-- Logs begin at Thu 2022-07-28 05:44:27 UTC, end at Thu 2022-07-28 06:56:22 UTC. -- Jul 28 06:22:32.598183 dv-cinder-850605-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 06:22:32.605688 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 06:22:32.607680 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 06:22:32 2022] *** Jul 28 06:22:32.607680 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 06:22:32.607680 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 06:22:32.607680 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: nodename: dv-cinder-850605-1-iscsi Jul 28 06:22:32.607680 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: machine: x86_64 Jul 28 06:22:32.607680 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: clock source: unix Jul 28 06:22:32.607680 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: pcre jit disabled Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: detected number of CPU cores: 4 Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: current working directory: / Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: detected binary path: /usr/bin/uwsgi-core Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: your processes number limit is 47680 Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: your memory page size is 4096 bytes Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: detected max file descriptor number: 2048 Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: lock engine: pthread robust mutexes Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: thunder lock: enabled Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 06:22:32.608126 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 06:22:32.648822 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: Python main interpreter initialized at 0x55e7d13b99c0 Jul 28 06:22:32.648822 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: python threads support enabled Jul 28 06:22:32.648822 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: your server socket listen backlog is limited to 100 connections Jul 28 06:22:32.648822 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: your mercy for graceful operations on workers is 90 seconds Jul 28 06:22:32.649212 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 06:22:32.649283 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: *** Operational MODE: preforking *** Jul 28 06:22:32.649566 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: *** uWSGI is running in multiple interpreter mode *** Jul 28 06:22:32.649566 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: spawned uWSGI master process (pid: 80264) Jul 28 06:22:32.649775 dv-cinder-850605-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 06:22:32.652958 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: spawned uWSGI worker 1 (pid: 80276, cores: 1) Jul 28 06:22:32.652958 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: spawned uWSGI worker 2 (pid: 80277, cores: 1) Jul 28 06:22:32.652958 dv-cinder-850605-1-iscsi devstack@keystone.service[80264]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 06:22:38.171781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 06:22:38.171781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: from cryptography.utils import int_from_bytes Jul 28 06:22:38.172612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 06:22:38.172612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: from cryptography.utils import int_from_bytes Jul 28 06:22:38.172728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 06:22:38.172728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: from cryptography.utils import int_from_bytes Jul 28 06:22:38.173140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 06:22:38.173140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: from cryptography.utils import int_from_bytes Jul 28 06:22:38.222620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 06:22:38.222811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 06:22:38.223004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] command line args: [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 06:22:38.223265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 06:22:38.223348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 06:22:38.223432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ================================================================================ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 06:22:38.223505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 06:22:38.223650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] command line args: [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 06:22:38.223729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] admin_token = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.223913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_dir = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.223998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 06:22:38.224081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ================================================================================ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 06:22:38.224157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.224268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_source = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.224346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] admin_token = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.224446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] debug = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.224584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] config_dir = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.224669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.224893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.224958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] config_source = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.225020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.225155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] debug = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.225266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.225994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.226450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.226554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.226659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.226764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.226861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.226962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_config_append = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_dir = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.227923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_config_append = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_dir = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_file = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.228982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.229934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] publish_errors = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.230983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.231063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.231186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.231263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.231384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.231520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.231606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.231752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] standard_threads = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.231981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] publish_errors = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_journal = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_json = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.232909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_stderr = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.233025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_syslog = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.233100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.233215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.233320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.233506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.233585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] standard_threads = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.233744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.233827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.233988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.234067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.234168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.234246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.234348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_journal = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.234457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.234548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_json = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.234692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_stderr = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.234775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.234874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] use_syslog = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.234953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.235107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 06:22:38.235194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.235364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.235448 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.235605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.235689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.235860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.235949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.236093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.236172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.236339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.236464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.236571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.236661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.236805 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.236887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.237048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.237126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.237285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.237398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.238074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.238185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.238329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.238556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.238634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.238817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.238897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.239020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.239137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.239283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.239362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.239535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.239615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.239782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.239860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.240950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.241997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.242165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.242245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.242402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.242482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.242649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.242734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.242879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.242966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.243111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.243189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.243343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.243421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.243571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.243661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.243799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.243878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.244953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.245986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.246140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.246217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.246368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.246453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.246605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.246684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.246840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.246945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.247076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.247188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.247311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.247415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.247545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.247648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.247781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.247872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248024 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.248970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249164 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249418 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.249969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.250143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.250227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.250380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.250460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.250619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.250697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.250858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.250942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.251089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.251210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.251321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.251407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.251558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.251650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.251804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.251884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.252973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.253054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.253836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.253976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.254064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.254190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.254300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.254387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.254547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.254628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.254787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.254867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.255980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.256976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.257132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.257225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.257386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.257468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.257628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.257718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.257863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.257944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.258109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.258189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.258347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.258427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.258582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.258660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.258820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.258901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.259055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.259142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.259293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.external = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.259373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.259524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.259605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.259770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.259849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.password = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.token = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.260951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.261041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.261194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.261275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.261453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.261534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.external = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.261701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.261780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.261947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.password = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] auth.token = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.262979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.263141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.263229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.263375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.263454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.263615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.263700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.263847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.263933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.264107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.264193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.264345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.264433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.264579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.264667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.264832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.264911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.265087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.265167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.265363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.265461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.265609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.265690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.265852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.265932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.266088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.266175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.266330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.266415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.266567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.266647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.266812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.266900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.267059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.267147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.267289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.267369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.267534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.267623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.267771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.267851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.268032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.268118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.268270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.268358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.268524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.268604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.268763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.268874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.269033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.269181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.269353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.269465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.269599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.269704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.269850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.269943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.270086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.270191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.270322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.270451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.270570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.270725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.270802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.270943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.271030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.271188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.271269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.271456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.271573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.271699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.271791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.271941 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.272992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.273145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.273225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.273412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.273508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.273671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.273752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.273919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.273998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.274154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.274242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.274384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.274464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.274619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.274698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.274846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.274933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.275081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.275161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.275320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.275401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.275557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.275645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.275804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.275885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.276035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.276114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.276281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.276369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.276527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.276607 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.276760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.276840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.277961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.278990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.279147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.279238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.279396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.279476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.279642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.279732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.279872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.279961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.280101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.280181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.280334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.280413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.280581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.280670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.280812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.280891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.281044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.281124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.281277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.281394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.281534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.281621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.281773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.281853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.282974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.283985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.284137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.284217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.284372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.284493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.284640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.284769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.284904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.285015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.285108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.285380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.285466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.285641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.285734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.285883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.285964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.286121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.286200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.286361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.286449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.286601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.286680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.286840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.286938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.287075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.287167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.287319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.287399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.287567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.287649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.287804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.287896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.288994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.289076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.289233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.289372 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.289499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.289580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.289738 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.289857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.289968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.290115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.290197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.290373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.290454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.290620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.290700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.290881 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.291007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.291119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.291201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.291369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.291449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.291611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.291706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.291844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.292847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.293013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.293093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.293241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.293340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.293514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.293594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.293764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.293847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.294003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.294083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.294247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.294325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.294492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.294573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.294730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.294809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295305 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295672 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] role.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.295995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] role.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.296148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.296226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.296391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.296478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.296645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.296793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.296907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.297030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.297160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.297277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.297452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.297549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.297674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.297796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.297926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.298043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.298176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.298276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.298411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.298529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.298661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.298779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.298913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.299017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.299165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.299245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.299418 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.299511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.299679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.299824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.299929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.300617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.300707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.300876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.301130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.301209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.301406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.301488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.301661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.301755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.301898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.301977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.302135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.302215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.302373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.302462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.302625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.302704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.302861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.302940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.303099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.303179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.303334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.303414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.303571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.303650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.303816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.303897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.304056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.304136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.304303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.304383 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.304542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.304622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.304778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.304857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.305995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.306077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.306251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.306329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.306495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.306574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.306745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.306834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.306991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.307071 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.307225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.307304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.307464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.307544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.307697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.307778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.307932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.308955 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.309120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.309201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.309374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.309458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.309622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.309713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.309855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.309945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.310953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 06:22:38.311065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.311308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.311558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.311799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.312050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.312290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.312544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.312786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.313037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.313272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.313537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.313776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 06:22:38.313947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 06:22:38.341224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.341453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.341679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.346542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.347002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.347398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.350633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.357078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.381122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.381373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.385749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.385985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.434182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.434508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.484987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.485220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.486045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.486288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.550368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.550693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.550903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.551995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.575155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.575388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.575536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.575627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.575770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.575856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.575998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.576092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.576209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.576292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.576419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.576503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.576630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.576846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.587615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.587962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.600699 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.600953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.601159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.601281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.601409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.601546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.601649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.601774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.601857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.601994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.602075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.602210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.602424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.602639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80276) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 06:22:38.619108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.619315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 06:22:38.619483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.619646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.619737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 06:22:38.619845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 06:22:38.620020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.620105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.620223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 06:22:38.620353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.620495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.620603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.620760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.620843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.621017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.621277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.631100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.631370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.631568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.631654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.631853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.631939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.632122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.632404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.638718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.639338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.643331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.643608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.644482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.644723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.644873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.645020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.645151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.645281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.645440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.645571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.645710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.645846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.645975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.646107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.646236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.646378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.646519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.646654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.646782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.646915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.647091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.647354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.669171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.670231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.670669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.670943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.671744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.672010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.675624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.675900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.676178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.676869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.677153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.677464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.682732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.683008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.684050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.684373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.687080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.687371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.687645 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.687918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.688183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.688448 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.688720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.688849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.688980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.689119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.689259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.689590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.689843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.690125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.690404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.691294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.705463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.705735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.708350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.708629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.710220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.712568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.713347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.713639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.715751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.716057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.717977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.720434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.721168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.724283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.725120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.725444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.725727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.726018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.726294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.726567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.727519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.727820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.728111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.728411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.728692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.728980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.743056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.743333 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.743593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.743855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.745677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.745962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.746227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.746497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.750643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.752161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.752430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.754186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.755615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.755899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.756171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.756447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.756740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.757029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.757311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.758722 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.759001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.759296 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.759577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.759869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.768527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.769412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.769864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.770229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.770503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.770779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.771048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.771940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.772221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.772506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.772793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.773069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.773366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.773643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.786432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.789243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.789362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.792253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.792337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.794689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.795248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.796310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.796589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.796864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.797741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.799376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.799663 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.799940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.803775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.805372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.807056 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.808254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.808539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.808656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.808825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.809093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.811593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.811884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.812168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.812454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.817644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.817923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.818198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.821419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.821708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.821992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.825190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.825484 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.825764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.826033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.826317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.826580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.826845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.827126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.827491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.827758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.828039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.828312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.828574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.828829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.829101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80276) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 06:22:38.829402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.829705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.829965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.830245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.830514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.830796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.831066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.831797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.831797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.832931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.832931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.832931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.832931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80276) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 06:22:38.862210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e7d13b99c0 pid: 80277 (default app) Jul 28 06:22:38.866042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb624-12e5-43c5-8465-db3eb5b96dec None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:38.866329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb624-12e5-43c5-8465-db3eb5b96dec None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:38.866602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb624-12e5-43c5-8465-db3eb5b96dec None None] PATH_INFO: `/v3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:38.866703 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e7d13b99c0 pid: 80276 (default app) Jul 28 06:22:38.870871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Thu Jul 28 06:22:38 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:46.520953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb7f4b-0b2e-4028-8c63-55b13e5b1538 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:46.521499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb7f4b-0b2e-4028-8c63-55b13e5b1538 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:46.521600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb7f4b-0b2e-4028-8c63-55b13e5b1538 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:46.525986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:22:46.531343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:46.531755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:46.531895 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:46.602910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: INFO dbcounter [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] Registered counter for database keystone Jul 28 06:22:46.624123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG oslo_db.sqlalchemy.engines [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80277) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 06:22:46.631026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writer thread running {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 06:22:46.807863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG passlib.handlers.bcrypt [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80277) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 06:22:46.808290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG passlib.handlers.bcrypt [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80277) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 06:22:46.862203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:46.863827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:46.987242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 06:22:47.148877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb7f4b-0b2e-4028-8c63-55b13e5b1538 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:47.220994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: INFO dbcounter [None req-0ceb7f4b-0b2e-4028-8c63-55b13e5b1538 None None] Registered counter for database keystone Jul 28 06:22:47.240651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG oslo_db.sqlalchemy.engines [None req-0ceb7f4b-0b2e-4028-8c63-55b13e5b1538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80276) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 06:22:47.246828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writer thread running {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 06:22:47.367815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb7f4b-0b2e-4028-8c63-55b13e5b1538 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:47.372190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7638-330f-404e-abab-ddd433c2539a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:47.372837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7638-330f-404e-abab-ddd433c2539a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:47.373122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7638-330f-404e-abab-ddd433c2539a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:47.373429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7638-330f-404e-abab-ddd433c2539a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:47.382865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7638-330f-404e-abab-ddd433c2539a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:47.440931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ed2e7638-330f-404e-abab-ddd433c2539a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 06:22:47.441216 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7638-330f-404e-abab-ddd433c2539a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:47.452393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Thu Jul 28 06:22:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:22:47.458478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:47.472727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c4bb4-3e1d-4910-8fbf-e70694431b0f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:47.477874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9232-0697-4685-9704-548e1ea4afa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:47.478986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9232-0697-4685-9704-548e1ea4afa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:47.479301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9232-0697-4685-9704-548e1ea4afa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:47.479643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9232-0697-4685-9704-548e1ea4afa2 admin admin] PATH_INFO: `/v3/projects/25016d71f554431cbc0d0222ca313083` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:47.481150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b9232-0697-4685-9704-548e1ea4afa2 admin admin] RBAC: Authorizing `identity:get_project(project_id=25016d71f554431cbc0d0222ca313083)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:47.531414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6c0b9232-0697-4685-9704-548e1ea4afa2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 06:22:47.531933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b9232-0697-4685-9704-548e1ea4afa2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:47.537122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Jul 28 06:22:47 2022] GET /identity/v3/projects/25016d71f554431cbc0d0222ca313083 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:49.538473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4408-dcf8-4b2f-ac01-68f8bd7e5c47 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:49.538987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4408-dcf8-4b2f-ac01-68f8bd7e5c47 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:49.539075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c4408-dcf8-4b2f-ac01-68f8bd7e5c47 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:49.543177 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:22:49.548644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a69cb6-4022-47e0-9d87-0bb79d65afa7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:49.549047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a69cb6-4022-47e0-9d87-0bb79d65afa7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:49.549189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a69cb6-4022-47e0-9d87-0bb79d65afa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:49.569978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e4a69cb6-4022-47e0-9d87-0bb79d65afa7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:49.571479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e4a69cb6-4022-47e0-9d87-0bb79d65afa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:49.579520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 06:22:49.601590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c4408-dcf8-4b2f-ac01-68f8bd7e5c47 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:49.613777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c4408-dcf8-4b2f-ac01-68f8bd7e5c47 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:49.618791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5d088-8b86-416d-b1a3-7840aeaf5277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:49.619407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5d088-8b86-416d-b1a3-7840aeaf5277 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:49.619692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5d088-8b86-416d-b1a3-7840aeaf5277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:49.619970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5d088-8b86-416d-b1a3-7840aeaf5277 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:49.627660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5d088-8b86-416d-b1a3-7840aeaf5277 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:49.629463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5d088-8b86-416d-b1a3-7840aeaf5277 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:49.638807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Thu Jul 28 06:22:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:22:49.644051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a69cb6-4022-47e0-9d87-0bb79d65afa7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:49.653285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a69cb6-4022-47e0-9d87-0bb79d65afa7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:49.657634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c48ae-ba64-4caa-bc01-237daea36be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:49.658267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c48ae-ba64-4caa-bc01-237daea36be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:49.658543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c48ae-ba64-4caa-bc01-237daea36be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:49.658817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c48ae-ba64-4caa-bc01-237daea36be5 admin admin] PATH_INFO: `/v3/users/5666224e1bf547bc87b970165a115158` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:49.660167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c48ae-ba64-4caa-bc01-237daea36be5 admin admin] RBAC: Authorizing `identity:get_user(user_id=5666224e1bf547bc87b970165a115158)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:49.662651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c48ae-ba64-4caa-bc01-237daea36be5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:49.664176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Thu Jul 28 06:22:49 2022] GET /identity/v3/users/5666224e1bf547bc87b970165a115158 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:51.736839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645cb1b-2694-4b39-81a3-3e37e936c1a7 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:51.737360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645cb1b-2694-4b39-81a3-3e37e936c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:51.737461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645cb1b-2694-4b39-81a3-3e37e936c1a7 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:51.738775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:51.744362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37804690-53be-4000-9397-9fe0609d184c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:51.744786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37804690-53be-4000-9397-9fe0609d184c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:51.744925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37804690-53be-4000-9397-9fe0609d184c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:51.764734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-37804690-53be-4000-9397-9fe0609d184c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:51.765950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-37804690-53be-4000-9397-9fe0609d184c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:51.834628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c48859-9785-4552-996c-72cfa2f35ec1 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:51.834908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c48859-9785-4552-996c-72cfa2f35ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:51.835191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c48859-9785-4552-996c-72cfa2f35ec1 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:51.836506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:22:51.841652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:51.841976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:51.842258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:51.846268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:22:51.869627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37804690-53be-4000-9397-9fe0609d184c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:51.885723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG passlib.handlers.bcrypt [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80276) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 06:22:51.886148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG passlib.handlers.bcrypt [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80276) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 06:22:51.893420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:51.932020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:51.933630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:51.973249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37804690-53be-4000-9397-9fe0609d184c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:52.007613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a54c3-1bd5-4637-9631-b286dcec7e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:52.008193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a54c3-1bd5-4637-9631-b286dcec7e0a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:52.008483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a54c3-1bd5-4637-9631-b286dcec7e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:52.008766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a54c3-1bd5-4637-9631-b286dcec7e0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:52.020035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a54c3-1bd5-4637-9631-b286dcec7e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1e9a54c3-1bd5-4637-9631-b286dcec7e0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:22:52.029324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:22:52.030256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:22:52.031054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 7/13] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:22:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:22:52.034165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 7/14] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:22:52.037097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a54c3-1bd5-4637-9631-b286dcec7e0a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:52.057069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:52.118367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:52.137571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a54c3-1bd5-4637-9631-b286dcec7e0a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:52.159233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6484f24-eba3-4dce-bf1c-fc6620348350 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:52.161718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:52.171719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:52.172340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:52.172641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:52.172922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:52.173679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:22:52.173992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:52.175465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:52.188938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:22:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:52.194980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:52.198846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a04b-ace3-4cac-820e-5d380d6e3e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:52.199995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a04b-ace3-4cac-820e-5d380d6e3e35 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:52.200281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a04b-ace3-4cac-820e-5d380d6e3e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:52.200559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a04b-ace3-4cac-820e-5d380d6e3e35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:52.273837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76a04b-ace3-4cac-820e-5d380d6e3e35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:52.275311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76a04b-ace3-4cac-820e-5d380d6e3e35 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:52.382579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:22:52.388092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a04b-ace3-4cac-820e-5d380d6e3e35 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:52.390786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:52.403642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5379-dce6-4a4a-9524-e888800681c2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:52.441902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef6f2-1a2e-4fb0-80cb-c8251800d112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:52.443537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef6f2-1a2e-4fb0-80cb-c8251800d112 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:52.444008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef6f2-1a2e-4fb0-80cb-c8251800d112 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:52.444288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef6f2-1a2e-4fb0-80cb-c8251800d112 None admin] PATH_INFO: `/v3/users/5666224e1bf547bc87b970165a115158` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:52.445518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ef6f2-1a2e-4fb0-80cb-c8251800d112 None admin] RBAC: Authorizing `identity:get_user(user_id=5666224e1bf547bc87b970165a115158)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:52.446992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ef6f2-1a2e-4fb0-80cb-c8251800d112 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:52.448592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:22:52 2022] GET /identity/v3/users/5666224e1bf547bc87b970165a115158 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:22:52.454357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef6f2-1a2e-4fb0-80cb-c8251800d112 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:52.470171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:52.491620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a04b-ace3-4cac-820e-5d380d6e3e35 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:52.526649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:52.527432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:52.527713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:52.527989 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:52.545626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:52.547095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:52.557155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef6f2-1a2e-4fb0-80cb-c8251800d112 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:22:52.565156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:22:52.566031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:22:52.566841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:22:52.566841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:22:52.566841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:22:52.566841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:22:52.566841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:22:52.572653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:52.592560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696e322-9493-4163-91fe-e123982152be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:52.593204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696e322-9493-4163-91fe-e123982152be None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:52.593520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696e322-9493-4163-91fe-e123982152be None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:52.593796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696e322-9493-4163-91fe-e123982152be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:52.594936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696e322-9493-4163-91fe-e123982152be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:52.596383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696e322-9493-4163-91fe-e123982152be None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:52.598010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:22:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:22:52.603907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696e322-9493-4163-91fe-e123982152be None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:52.629063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:52.673106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a238054-4aec-46a2-b897-ecfbfcaf12b9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:52.686907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:52.706658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696e322-9493-4163-91fe-e123982152be None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:52.707204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b9506-5c10-438a-ae74-ef82c62764a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:52.707990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b9506-5c10-438a-ae74-ef82c62764a2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:52.708352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b9506-5c10-438a-ae74-ef82c62764a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:52.708629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b9506-5c10-438a-ae74-ef82c62764a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:52.709357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b9506-5c10-438a-ae74-ef82c62764a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:22:52.709662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b9506-5c10-438a-ae74-ef82c62764a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:52.711033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b9506-5c10-438a-ae74-ef82c62764a2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:52.728502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:22:52.742002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29eb7d1-0eba-4447-be4d-3290083dbbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:52.742659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb7d1-0eba-4447-be4d-3290083dbbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:52.742946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb7d1-0eba-4447-be4d-3290083dbbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:52.743222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb7d1-0eba-4447-be4d-3290083dbbd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:52.744046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29eb7d1-0eba-4447-be4d-3290083dbbd3 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.domain.id=default, user.id=5666224e1bf547bc87b970165a115158) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:22:52.744362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29eb7d1-0eba-4447-be4d-3290083dbbd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:52.745910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29eb7d1-0eba-4447-be4d-3290083dbbd3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:52.753951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Thu Jul 28 06:22:52 2022] GET /identity/v3/role_assignments?user.id=5666224e1bf547bc87b970165a115158&scope.domain.id=default&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:22:55.130890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562b92-9ba8-4798-8703-9c7e1ebf2c79 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.130890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562b92-9ba8-4798-8703-9c7e1ebf2c79 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.131582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562b92-9ba8-4798-8703-9c7e1ebf2c79 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.132829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:55.137896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc122f5e-9b5b-4c18-b1c6-44569bcbd671 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.138669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc122f5e-9b5b-4c18-b1c6-44569bcbd671 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.139100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc122f5e-9b5b-4c18-b1c6-44569bcbd671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.167054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fc122f5e-9b5b-4c18-b1c6-44569bcbd671 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:55.168464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fc122f5e-9b5b-4c18-b1c6-44569bcbd671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:55.269785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:22:55.293846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aacc2-fb7c-429f-b62b-d6842b27d731 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.294349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aacc2-fb7c-429f-b62b-d6842b27d731 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.294800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aacc2-fb7c-429f-b62b-d6842b27d731 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.296290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:55.302572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab3f-4ee9-49b8-b11c-ca94f5c08bf1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.303037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab3f-4ee9-49b8-b11c-ca94f5c08bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.303496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab3f-4ee9-49b8-b11c-ca94f5c08bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.305852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1aacc2-fb7c-429f-b62b-d6842b27d731 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:55.344363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0ef7ab3f-4ee9-49b8-b11c-ca94f5c08bf1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:55.346129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0ef7ab3f-4ee9-49b8-b11c-ca94f5c08bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:55.479380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:22:55.480286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1aacc2-fb7c-429f-b62b-d6842b27d731 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:55.480637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35de7c7-11f7-4058-a02a-6b9b5f829d64 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.480932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35de7c7-11f7-4058-a02a-6b9b5f829d64 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.481214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35de7c7-11f7-4058-a02a-6b9b5f829d64 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.482593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:55.491539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cba46-4892-48d4-9bb2-3ba5b4b5810c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.491830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cba46-4892-48d4-9bb2-3ba5b4b5810c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.492113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cba46-4892-48d4-9bb2-3ba5b4b5810c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.505717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:55.515390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:55.524174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f43cba46-4892-48d4-9bb2-3ba5b4b5810c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:55.526131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f43cba46-4892-48d4-9bb2-3ba5b4b5810c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:55.532975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:55.538032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.538546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.539038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.559078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:55.562660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:22:55.586974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:22:55.587878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:22:55.588687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:22:55.588687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:22:55.588687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:22:55.588687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:22:55.588687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 13/27] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:22:55.589520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:55.626507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 15/28] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:22:55.628329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7445a3-2ef6-40fc-8761-1b0430eb6923 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.628843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7445a3-2ef6-40fc-8761-1b0430eb6923 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.629339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7445a3-2ef6-40fc-8761-1b0430eb6923 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.632521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 16/29] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:55.633608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7445a3-2ef6-40fc-8761-1b0430eb6923 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:55.709421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f0bcc-1165-4be6-bf83-e5196722c833 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:55.751218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7445a3-2ef6-40fc-8761-1b0430eb6923 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:55.751704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39266ce3-0370-4fe0-8edf-b41d6561adad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:55.752589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39266ce3-0370-4fe0-8edf-b41d6561adad None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.753021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39266ce3-0370-4fe0-8edf-b41d6561adad None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.753543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39266ce3-0370-4fe0-8edf-b41d6561adad None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.755071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39266ce3-0370-4fe0-8edf-b41d6561adad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:55.756691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39266ce3-0370-4fe0-8edf-b41d6561adad None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:55.793659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53eb11d-2dc8-4430-aeba-7162c8b10050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:55.794306 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eb11d-2dc8-4430-aeba-7162c8b10050 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.794627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eb11d-2dc8-4430-aeba-7162c8b10050 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.794954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eb11d-2dc8-4430-aeba-7162c8b10050 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.795735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eb11d-2dc8-4430-aeba-7162c8b10050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:22:55.796045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eb11d-2dc8-4430-aeba-7162c8b10050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:55.797529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eb11d-2dc8-4430-aeba-7162c8b10050 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:55.800042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:55.827331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 17/30] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:55.827331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e965f2-5571-40c5-8c4c-a734d732e150 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:55.827331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e965f2-5571-40c5-8c4c-a734d732e150 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:55.827331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e965f2-5571-40c5-8c4c-a734d732e150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:55.844886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-52e965f2-5571-40c5-8c4c-a734d732e150 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:55.846132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-52e965f2-5571-40c5-8c4c-a734d732e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:55.925943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 18/31] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:22:55.928169 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e965f2-5571-40c5-8c4c-a734d732e150 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:56.045170 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e965f2-5571-40c5-8c4c-a734d732e150 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:56.054716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 14/32] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jul 28 06:22:55 2022] POST /identity/v3/roles => generated 219 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:22:56.055478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39266ce3-0370-4fe0-8edf-b41d6561adad None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:56.096284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:56.111023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:56.132641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14719b1-82f2-4d41-9073-a8676ca30d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:56.133333 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14719b1-82f2-4d41-9073-a8676ca30d74 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.133627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14719b1-82f2-4d41-9073-a8676ca30d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.133907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14719b1-82f2-4d41-9073-a8676ca30d74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.158118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14719b1-82f2-4d41-9073-a8676ca30d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c14719b1-82f2-4d41-9073-a8676ca30d74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:22:56.172079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:22:56.173008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:22:56.173818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:22:56.174914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 19/33] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:22:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:22:56.176096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14719b1-82f2-4d41-9073-a8676ca30d74 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:56.252074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39266ce3-0370-4fe0-8edf-b41d6561adad None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:56.303278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8283d9c-b76c-4870-b935-aa8f94dcf5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:56.304790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8283d9c-b76c-4870-b935-aa8f94dcf5e8 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.305092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8283d9c-b76c-4870-b935-aa8f94dcf5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.305458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8283d9c-b76c-4870-b935-aa8f94dcf5e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.306432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8283d9c-b76c-4870-b935-aa8f94dcf5e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:56.308040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8283d9c-b76c-4870-b935-aa8f94dcf5e8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:56.349430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14719b1-82f2-4d41-9073-a8676ca30d74 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:56.365022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 15/34] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:22:56 2022] POST /identity/v3/projects => generated 301 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:22:56.365947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8283d9c-b76c-4870-b935-aa8f94dcf5e8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:56.421077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a73037-64a0-443f-8920-d023ffd3289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:56.421742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a73037-64a0-443f-8920-d023ffd3289f None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.422027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a73037-64a0-443f-8920-d023ffd3289f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.422299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a73037-64a0-443f-8920-d023ffd3289f None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.423025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a73037-64a0-443f-8920-d023ffd3289f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:56.424510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a73037-64a0-443f-8920-d023ffd3289f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:56.487438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:56.518885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8283d9c-b76c-4870-b935-aa8f94dcf5e8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:56.565257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:56.565857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.566149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.566435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.567700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:22:56.568012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:56.569452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:56.585605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 16/35] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:22:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:56.594628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:56.627359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 20/36] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jul 28 06:22:56 2022] POST /identity/v3/roles => generated 225 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:22:56.632572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e6408-7abd-4cd6-99ce-fa229f4c95fb None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.633357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e6408-7abd-4cd6-99ce-fa229f4c95fb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.633938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e6408-7abd-4cd6-99ce-fa229f4c95fb None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.635598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 21/37] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:22:56.641450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553a24-2c8c-4fd8-96e4-23d2caf4fea7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.642046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553a24-2c8c-4fd8-96e4-23d2caf4fea7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.642572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553a24-2c8c-4fd8-96e4-23d2caf4fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.667748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a6553a24-2c8c-4fd8-96e4-23d2caf4fea7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:56.669834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a6553a24-2c8c-4fd8-96e4-23d2caf4fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:56.725865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33024c-85bc-4bcc-94aa-fe14d1953b35 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:56.740879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:56.769833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f6e08-93b7-49d7-8789-0fecc9d96ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:56.771083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6e08-93b7-49d7-8789-0fecc9d96ace None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.771379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6e08-93b7-49d7-8789-0fecc9d96ace None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.771670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6e08-93b7-49d7-8789-0fecc9d96ace None admin] PATH_INFO: `/v3/users/5666224e1bf547bc87b970165a115158` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.773517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f6e08-93b7-49d7-8789-0fecc9d96ace None admin] RBAC: Authorizing `identity:get_user(user_id=5666224e1bf547bc87b970165a115158)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:56.774796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f6e08-93b7-49d7-8789-0fecc9d96ace None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:56.776894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 17/38] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:22:56 2022] GET /identity/v3/users/5666224e1bf547bc87b970165a115158 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:56.782289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 22/39] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:22:56.785337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f6e08-93b7-49d7-8789-0fecc9d96ace None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:56.800652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:56.810120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6553a24-2c8c-4fd8-96e4-23d2caf4fea7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:56.902937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f6e08-93b7-49d7-8789-0fecc9d96ace None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:56.922978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6553a24-2c8c-4fd8-96e4-23d2caf4fea7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:56.936380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7aad2-4816-4beb-9d1a-71d087542ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:56.937033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7aad2-4816-4beb-9d1a-71d087542ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.937338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7aad2-4816-4beb-9d1a-71d087542ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.937624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7aad2-4816-4beb-9d1a-71d087542ae3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.938831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7aad2-4816-4beb-9d1a-71d087542ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:56.940289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7aad2-4816-4beb-9d1a-71d087542ae3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:56.941942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 18/40] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:22:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:56.943151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fb83a-1741-434e-88db-42e227b35af0 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.943444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fb83a-1741-434e-88db-42e227b35af0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.943727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fb83a-1741-434e-88db-42e227b35af0 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.945131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 19/41] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:56.950006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fb83a-1741-434e-88db-42e227b35af0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:56.958606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edde4f-6fc1-4cad-9f67-9253d15857ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:56.959232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edde4f-6fc1-4cad-9f67-9253d15857ac None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.959516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edde4f-6fc1-4cad-9f67-9253d15857ac None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.959793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edde4f-6fc1-4cad-9f67-9253d15857ac None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:56.960521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edde4f-6fc1-4cad-9f67-9253d15857ac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:56.962067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edde4f-6fc1-4cad-9f67-9253d15857ac None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:56.977392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 23/42] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jul 28 06:22:56 2022] POST /identity/v3/roles => generated 223 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:22:56.977392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82a105-4c39-4a13-8c82-37166ea85635 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:56.977392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82a105-4c39-4a13-8c82-37166ea85635 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:56.977392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82a105-4c39-4a13-8c82-37166ea85635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:57.008411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3a82a105-4c39-4a13-8c82-37166ea85635 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:57.009697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3a82a105-4c39-4a13-8c82-37166ea85635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:57.040702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:57.043434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:57.073918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fb83a-1741-434e-88db-42e227b35af0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:57.089994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 24/43] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:22:57.114775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82a105-4c39-4a13-8c82-37166ea85635 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:57.120037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd1c08-25a5-46ef-ba63-efba3cc44ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:57.120741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1c08-25a5-46ef-ba63-efba3cc44ea3 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:57.121026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1c08-25a5-46ef-ba63-efba3cc44ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:57.121316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1c08-25a5-46ef-ba63-efba3cc44ea3 None admin] PATH_INFO: `/v3/domains/default/users/5666224e1bf547bc87b970165a115158/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:57.123355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd1c08-25a5-46ef-ba63-efba3cc44ea3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5666224e1bf547bc87b970165a115158, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:57.124832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd1c08-25a5-46ef-ba63-efba3cc44ea3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:57.143132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 20/44] 10.222.0.20 () {58 vars in 1361 bytes} [Thu Jul 28 06:22:56 2022] PUT /identity/v3/domains/default/users/5666224e1bf547bc87b970165a115158/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:22:57.240334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82a105-4c39-4a13-8c82-37166ea85635 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:57.276485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7412ee-ed0d-47a1-8975-460ac4bce41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:57.277140 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7412ee-ed0d-47a1-8975-460ac4bce41b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:57.277460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7412ee-ed0d-47a1-8975-460ac4bce41b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:57.277741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7412ee-ed0d-47a1-8975-460ac4bce41b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:57.278937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7412ee-ed0d-47a1-8975-460ac4bce41b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:57.280417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7412ee-ed0d-47a1-8975-460ac4bce41b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:57.282092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 25/45] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:22:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:22:57.287893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd1c08-25a5-46ef-ba63-efba3cc44ea3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:57.390710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd1c08-25a5-46ef-ba63-efba3cc44ea3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:57.424242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc07cac-892a-4268-88cc-94a002988ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:57.424869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc07cac-892a-4268-88cc-94a002988ed3 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:57.425159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc07cac-892a-4268-88cc-94a002988ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:57.425478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc07cac-892a-4268-88cc-94a002988ed3 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:57.426184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc07cac-892a-4268-88cc-94a002988ed3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:57.427739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc07cac-892a-4268-88cc-94a002988ed3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:57.437713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:57.460365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 21/46] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:22:57 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:22:59.135566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06a33-bb8b-4217-89a2-721351ca6731 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.136081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06a33-bb8b-4217-89a2-721351ca6731 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.136163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06a33-bb8b-4217-89a2-721351ca6731 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.137497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 26/47] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:22:59.142775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fea47-35d0-4510-b461-4d4de55b0ac1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.143219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fea47-35d0-4510-b461-4d4de55b0ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.143309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fea47-35d0-4510-b461-4d4de55b0ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.163186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-858fea47-35d0-4510-b461-4d4de55b0ac1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:59.164692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-858fea47-35d0-4510-b461-4d4de55b0ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:59.230243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 22/48] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:22:59.252388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb06a33-bb8b-4217-89a2-721351ca6731 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:59.283919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:59.355477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb06a33-bb8b-4217-89a2-721351ca6731 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:59.388447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f3a71-22dd-438b-8a1e-d19ad1ac394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:59.389022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f3a71-22dd-438b-8a1e-d19ad1ac394f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.389327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f3a71-22dd-438b-8a1e-d19ad1ac394f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.389606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f3a71-22dd-438b-8a1e-d19ad1ac394f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.400206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f3a71-22dd-438b-8a1e-d19ad1ac394f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-be4f3a71-22dd-438b-8a1e-d19ad1ac394f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:22:59.405157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:22:59.406033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:22:59.406807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 27/49] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:22:59.412269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fea47-35d0-4510-b461-4d4de55b0ac1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:59.465954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62e059-9c3f-42eb-9f93-b9792f7a444a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.466245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62e059-9c3f-42eb-9f93-b9792f7a444a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.466522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62e059-9c3f-42eb-9f93-b9792f7a444a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.467878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 28/50] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:22:59.472709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc309b-493b-44c5-a8b4-42bc5ddb74ac None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.473005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc309b-493b-44c5-a8b4-42bc5ddb74ac None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.473586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc309b-493b-44c5-a8b4-42bc5ddb74ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.491527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-78fc309b-493b-44c5-a8b4-42bc5ddb74ac None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:22:59.492727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-78fc309b-493b-44c5-a8b4-42bc5ddb74ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:22:59.514881 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fea47-35d0-4510-b461-4d4de55b0ac1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:59.535337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:59.551937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:59.552568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.552853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.553142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.553909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:22:59.554219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:59.555604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:59.560322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 29/51] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:22:59.563629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 23/52] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:22:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:59.569616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc309b-493b-44c5-a8b4-42bc5ddb74ac None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:59.584133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:59.593902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:59.672743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc309b-493b-44c5-a8b4-42bc5ddb74ac None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:59.706469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf477-33bb-42b8-8762-8d476f353593 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:59.707331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884b4b5-56a2-429f-a682-e3fa05078d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:59.708688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884b4b5-56a2-429f-a682-e3fa05078d48 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.708977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884b4b5-56a2-429f-a682-e3fa05078d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.709258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884b4b5-56a2-429f-a682-e3fa05078d48 None admin] PATH_INFO: `/v3/users/5666224e1bf547bc87b970165a115158` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.710481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884b4b5-56a2-429f-a682-e3fa05078d48 None admin] RBAC: Authorizing `identity:get_user(user_id=5666224e1bf547bc87b970165a115158)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:59.711942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884b4b5-56a2-429f-a682-e3fa05078d48 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:59.713565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 30/53] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:22:59 2022] GET /identity/v3/users/5666224e1bf547bc87b970165a115158 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:22:59.718978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884b4b5-56a2-429f-a682-e3fa05078d48 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:59.741247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bda51-b642-4d2a-bd70-28412c41d89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:59.741925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bda51-b642-4d2a-bd70-28412c41d89c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.742218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bda51-b642-4d2a-bd70-28412c41d89c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.742504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bda51-b642-4d2a-bd70-28412c41d89c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.743685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bda51-b642-4d2a-bd70-28412c41d89c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:59.745137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bda51-b642-4d2a-bd70-28412c41d89c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:59.746770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 24/54] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:22:59.752328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bda51-b642-4d2a-bd70-28412c41d89c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:59.800880 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:59.807506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:22:59.818604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884b4b5-56a2-429f-a682-e3fa05078d48 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:59.852297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bda51-b642-4d2a-bd70-28412c41d89c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:22:59.852613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d960-c733-437d-a4db-bf8f8c8401ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:59.853259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d960-c733-437d-a4db-bf8f8c8401ba None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.853585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d960-c733-437d-a4db-bf8f8c8401ba None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.853863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d960-c733-437d-a4db-bf8f8c8401ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.855021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44d960-c733-437d-a4db-bf8f8c8401ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:59.856455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44d960-c733-437d-a4db-bf8f8c8401ba None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:59.858112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 31/55] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:22:59.864336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d960-c733-437d-a4db-bf8f8c8401ba None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:22:59.885935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f1cd3-b852-4470-9c37-8fa112ce9061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:22:59.886560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1cd3-b852-4470-9c37-8fa112ce9061 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:22:59.886895 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1cd3-b852-4470-9c37-8fa112ce9061 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:22:59.887161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f1cd3-b852-4470-9c37-8fa112ce9061 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:22:59.887841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1cd3-b852-4470-9c37-8fa112ce9061 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:22:59.889271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f1cd3-b852-4470-9c37-8fa112ce9061 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:22:59.917863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 25/56] 10.222.0.20 () {60 vars in 1140 bytes} [Thu Jul 28 06:22:59 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:22:59.983925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d960-c733-437d-a4db-bf8f8c8401ba None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:00.022338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8642d2-04eb-447c-b1b4-eed59a62681d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:00.022980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8642d2-04eb-447c-b1b4-eed59a62681d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:00.023264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8642d2-04eb-447c-b1b4-eed59a62681d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:00.023542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8642d2-04eb-447c-b1b4-eed59a62681d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:00.024343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8642d2-04eb-447c-b1b4-eed59a62681d None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.domain.id=default, user.id=5666224e1bf547bc87b970165a115158) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:00.024659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8642d2-04eb-447c-b1b4-eed59a62681d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:00.026158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8642d2-04eb-447c-b1b4-eed59a62681d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:00.032741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 32/57] 10.222.0.20 () {56 vars in 1322 bytes} [Thu Jul 28 06:22:59 2022] GET /identity/v3/role_assignments?user.id=5666224e1bf547bc87b970165a115158&scope.domain.id=default&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 530 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:01.957368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193660e-973b-4230-8825-52f08109669d None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:01.958072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193660e-973b-4230-8825-52f08109669d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:01.958072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193660e-973b-4230-8825-52f08109669d None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:01.959381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 26/58] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:01.964528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214728-9cae-4601-a7e1-0c65fca12f9f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:01.964900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214728-9cae-4601-a7e1-0c65fca12f9f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:01.965089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214728-9cae-4601-a7e1-0c65fca12f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:01.983247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cd214728-9cae-4601-a7e1-0c65fca12f9f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:01.984445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cd214728-9cae-4601-a7e1-0c65fca12f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:02.001171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:02.051255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 33/59] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:02.075015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193660e-973b-4230-8825-52f08109669d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:02.177193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193660e-973b-4230-8825-52f08109669d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:02.178701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:02.212155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9a5dd-233a-4022-8aab-ad7e5112f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:02.212731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9a5dd-233a-4022-8aab-ad7e5112f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:02.213016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9a5dd-233a-4022-8aab-ad7e5112f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:02.213321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9a5dd-233a-4022-8aab-ad7e5112f9a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:02.214472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9a5dd-233a-4022-8aab-ad7e5112f9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:02.215920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9a5dd-233a-4022-8aab-ad7e5112f9a2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:02.217516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 27/60] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:02.223038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd214728-9cae-4601-a7e1-0c65fca12f9f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:02.319110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd214728-9cae-4601-a7e1-0c65fca12f9f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:02.351896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37394988-888f-4959-98a3-70a7d6f8365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:02.352508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394988-888f-4959-98a3-70a7d6f8365a None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:02.352795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394988-888f-4959-98a3-70a7d6f8365a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:02.353074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394988-888f-4959-98a3-70a7d6f8365a None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:02.353797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37394988-888f-4959-98a3-70a7d6f8365a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:02.355221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37394988-888f-4959-98a3-70a7d6f8365a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:02.382044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 34/61] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:02 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:05.457215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ccbba-4037-4d04-bfb5-0da82955e332 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:05.457768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ccbba-4037-4d04-bfb5-0da82955e332 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:05.457842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ccbba-4037-4d04-bfb5-0da82955e332 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:05.459153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 28/62] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:05.464437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990d8de-a97e-4bc7-95ba-c6e6df5ae3a0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:05.465206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990d8de-a97e-4bc7-95ba-c6e6df5ae3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:05.465724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990d8de-a97e-4bc7-95ba-c6e6df5ae3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:05.494581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9990d8de-a97e-4bc7-95ba-c6e6df5ae3a0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:05.495998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9990d8de-a97e-4bc7-95ba-c6e6df5ae3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:05.498293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:05.598319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 35/63] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:05.634190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ccbba-4037-4d04-bfb5-0da82955e332 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:05.663156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c1e1-d47c-435e-8322-ac3fc0adde73 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:05.663439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c1e1-d47c-435e-8322-ac3fc0adde73 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:05.663718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c1e1-d47c-435e-8322-ac3fc0adde73 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:05.665067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 36/64] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:05.677656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ef792-640d-4198-9cdb-a967ee87d8b5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:05.677945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ef792-640d-4198-9cdb-a967ee87d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:05.678224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ef792-640d-4198-9cdb-a967ee87d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:05.710844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d14ef792-640d-4198-9cdb-a967ee87d8b5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:05.711961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d14ef792-640d-4198-9cdb-a967ee87d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:05.819692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ccbba-4037-4d04-bfb5-0da82955e332 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:05.839030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 37/65] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:05.840580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b0cd8-7ee3-459a-b8e3-848d958564bc None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:05.841018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b0cd8-7ee3-459a-b8e3-848d958564bc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:05.841470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b0cd8-7ee3-459a-b8e3-848d958564bc None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:05.843148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 38/66] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:05.848854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ced5-73f9-44f3-a013-7248b6b8d987 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:05.849445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ced5-73f9-44f3-a013-7248b6b8d987 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:05.849894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ced5-73f9-44f3-a013-7248b6b8d987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:05.868689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-35d3ced5-73f9-44f3-a013-7248b6b8d987 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:05.870201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-35d3ced5-73f9-44f3-a013-7248b6b8d987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:05.882425 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e6e3-d146-4fc8-901b-57a55be56d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:05.883075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e6e3-d146-4fc8-901b-57a55be56d42 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:05.883351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e6e3-d146-4fc8-901b-57a55be56d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:05.883868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e6e3-d146-4fc8-901b-57a55be56d42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:05.897322 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6e6e3-d146-4fc8-901b-57a55be56d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-f7e6e6e3-d146-4fc8-901b-57a55be56d42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:05.904563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:05.905652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 29/67] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:05.906526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e6e3-d146-4fc8-901b-57a55be56d42 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:05.941564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:05.942929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:05.949709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 39/68] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:05.950713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3ced5-73f9-44f3-a013-7248b6b8d987 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.016265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e6e3-d146-4fc8-901b-57a55be56d42 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.053262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093122dc-8d1c-4f29-9b5e-96089f082493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.053942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093122dc-8d1c-4f29-9b5e-96089f082493 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.054233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093122dc-8d1c-4f29-9b5e-96089f082493 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.054512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093122dc-8d1c-4f29-9b5e-96089f082493 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.065516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093122dc-8d1c-4f29-9b5e-96089f082493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:06.068685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3ced5-73f9-44f3-a013-7248b6b8d987 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-093122dc-8d1c-4f29-9b5e-96089f082493 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:06.071767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:06.072603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:06.073412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 30/69] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:06.074306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093122dc-8d1c-4f29-9b5e-96089f082493 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.112420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c47446-0c3b-4b3e-990d-31ba90ef2f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.114915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c47446-0c3b-4b3e-990d-31ba90ef2f5d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.115352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c47446-0c3b-4b3e-990d-31ba90ef2f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.115795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c47446-0c3b-4b3e-990d-31ba90ef2f5d None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.116661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c47446-0c3b-4b3e-990d-31ba90ef2f5d None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:06.117101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c47446-0c3b-4b3e-990d-31ba90ef2f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:06.118647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c47446-0c3b-4b3e-990d-31ba90ef2f5d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:06.130887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 40/70] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:06.131326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267c740-326b-4532-8412-ad8febaf3987 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.131670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267c740-326b-4532-8412-ad8febaf3987 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.131987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267c740-326b-4532-8412-ad8febaf3987 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.137225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 41/71] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:06.137918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267c740-326b-4532-8412-ad8febaf3987 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.177951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:06.189498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093122dc-8d1c-4f29-9b5e-96089f082493 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.206907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:06.225768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f77dfd-9299-4566-9810-ffc9520b2e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.226437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f77dfd-9299-4566-9810-ffc9520b2e52 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.226724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f77dfd-9299-4566-9810-ffc9520b2e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.227018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f77dfd-9299-4566-9810-ffc9520b2e52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.238031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f77dfd-9299-4566-9810-ffc9520b2e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-19f77dfd-9299-4566-9810-ffc9520b2e52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:06.243715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:06.244584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:06.245628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 31/72] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:06.246284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f77dfd-9299-4566-9810-ffc9520b2e52 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.260659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267c740-326b-4532-8412-ad8febaf3987 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.298320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae1222-5ff7-4e3b-8066-ab9a5189f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.298866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae1222-5ff7-4e3b-8066-ab9a5189f57b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.299147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae1222-5ff7-4e3b-8066-ab9a5189f57b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.299422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae1222-5ff7-4e3b-8066-ab9a5189f57b None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.300134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae1222-5ff7-4e3b-8066-ab9a5189f57b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:06.300435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae1222-5ff7-4e3b-8066-ab9a5189f57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:06.301876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae1222-5ff7-4e3b-8066-ab9a5189f57b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:06.309850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 42/73] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:06.310989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9994f4-3a71-49a5-83b5-2dcc0b007cdc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.311274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9994f4-3a71-49a5-83b5-2dcc0b007cdc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.311549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9994f4-3a71-49a5-83b5-2dcc0b007cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.331916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9e9994f4-3a71-49a5-83b5-2dcc0b007cdc None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:06.333782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9e9994f4-3a71-49a5-83b5-2dcc0b007cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:06.349987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f77dfd-9299-4566-9810-ffc9520b2e52 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.384686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d515db-6f76-4bb8-9998-e5ed1843ef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.385359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d515db-6f76-4bb8-9998-e5ed1843ef46 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.385650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d515db-6f76-4bb8-9998-e5ed1843ef46 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.385934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d515db-6f76-4bb8-9998-e5ed1843ef46 None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.399974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 43/74] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:06.400808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9994f4-3a71-49a5-83b5-2dcc0b007cdc None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.411653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d515db-6f76-4bb8-9998-e5ed1843ef46 None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:06.413109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d515db-6f76-4bb8-9998-e5ed1843ef46 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:06.419104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 32/75] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:06.419104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be8c4b-4c60-40b5-ada7-ebe687cde781 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.419104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be8c4b-4c60-40b5-ada7-ebe687cde781 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.419104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be8c4b-4c60-40b5-ada7-ebe687cde781 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.419104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 33/76] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:06.419104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be8c4b-4c60-40b5-ada7-ebe687cde781 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.433151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:06.462785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:06.508944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9994f4-3a71-49a5-83b5-2dcc0b007cdc None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.530314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be8c4b-4c60-40b5-ada7-ebe687cde781 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.543204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.543825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.544109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.544387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.545121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:06.545461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:06.546838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:06.555431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 44/77] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:06.555987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.564619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75597a9-d03a-43c2-b7ae-f9d1406ddd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.565259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75597a9-d03a-43c2-b7ae-f9d1406ddd83 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.565572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75597a9-d03a-43c2-b7ae-f9d1406ddd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.565847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75597a9-d03a-43c2-b7ae-f9d1406ddd83 None admin] PATH_INFO: `/v3/users/5666224e1bf547bc87b970165a115158` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.567555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75597a9-d03a-43c2-b7ae-f9d1406ddd83 None admin] RBAC: Authorizing `identity:get_user(user_id=5666224e1bf547bc87b970165a115158)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:06.569147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75597a9-d03a-43c2-b7ae-f9d1406ddd83 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:06.570762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 34/78] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/users/5666224e1bf547bc87b970165a115158 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:06.571507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75597a9-d03a-43c2-b7ae-f9d1406ddd83 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.652614 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:06.659084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2459724-a594-4ee0-9ac3-e432e7b23126 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.673767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:06.678095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75597a9-d03a-43c2-b7ae-f9d1406ddd83 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.696009 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda78ec-e4fe-4b7a-a5c0-d81b21b5a2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.696635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda78ec-e4fe-4b7a-a5c0-d81b21b5a2fc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.696924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda78ec-e4fe-4b7a-a5c0-d81b21b5a2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.697204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda78ec-e4fe-4b7a-a5c0-d81b21b5a2fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.713396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bff869-bb4f-40a3-8e0b-741fea8f332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.714115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff869-bb4f-40a3-8e0b-741fea8f332e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.714406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff869-bb4f-40a3-8e0b-741fea8f332e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.714684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bff869-bb4f-40a3-8e0b-741fea8f332e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.724606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bff869-bb4f-40a3-8e0b-741fea8f332e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-89bff869-bb4f-40a3-8e0b-741fea8f332e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:06.729595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:06.730433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:06.731242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 35/79] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:06.731903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b28f64-aac5-49b6-89e1-6f9f23fd5745 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.732207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b28f64-aac5-49b6-89e1-6f9f23fd5745 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.732458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b28f64-aac5-49b6-89e1-6f9f23fd5745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.750300 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c5b28f64-aac5-49b6-89e1-6f9f23fd5745 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:06.751509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c5b28f64-aac5-49b6-89e1-6f9f23fd5745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:06.769451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda78ec-e4fe-4b7a-a5c0-d81b21b5a2fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:06.771463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda78ec-e4fe-4b7a-a5c0-d81b21b5a2fc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:06.819794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 36/80] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:06.820548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b28f64-aac5-49b6-89e1-6f9f23fd5745 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.870617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 45/81] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:06.871458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda78ec-e4fe-4b7a-a5c0-d81b21b5a2fc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.887674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:06.897972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:06.935941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b28f64-aac5-49b6-89e1-6f9f23fd5745 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:06.970113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae001-efe0-4459-b2bb-f5e3ac31903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:06.970757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae001-efe0-4459-b2bb-f5e3ac31903d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:06.971043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae001-efe0-4459-b2bb-f5e3ac31903d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:06.971330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae001-efe0-4459-b2bb-f5e3ac31903d None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:06.972507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eae001-efe0-4459-b2bb-f5e3ac31903d None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:06.975108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eae001-efe0-4459-b2bb-f5e3ac31903d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:06.976873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 37/82] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:06.977715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae001-efe0-4459-b2bb-f5e3ac31903d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:06.977813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda78ec-e4fe-4b7a-a5c0-d81b21b5a2fc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.014229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6beebad-be8a-40f3-8ae1-d3196043c759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.014851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6beebad-be8a-40f3-8ae1-d3196043c759 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.015136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6beebad-be8a-40f3-8ae1-d3196043c759 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.015411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6beebad-be8a-40f3-8ae1-d3196043c759 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.075814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae001-efe0-4459-b2bb-f5e3ac31903d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.082102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6beebad-be8a-40f3-8ae1-d3196043c759 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.083631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6beebad-be8a-40f3-8ae1-d3196043c759 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.097465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:07.108739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.109385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.109673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.109949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.110659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:07.110968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.112333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.114182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:07.121263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 38/83] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:07.122196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.186096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 46/84] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:07.186759 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6beebad-be8a-40f3-8ae1-d3196043c759 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.226981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb599b3-de3e-43b0-85ca-e4873969addd None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.262864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286f965-696c-417c-896b-5108ebcc3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.263509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286f965-696c-417c-896b-5108ebcc3ebf None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.263799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286f965-696c-417c-896b-5108ebcc3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.264079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286f965-696c-417c-896b-5108ebcc3ebf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.265246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286f965-696c-417c-896b-5108ebcc3ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.266741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286f965-696c-417c-896b-5108ebcc3ebf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.268327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 39/85] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:07.269173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286f965-696c-417c-896b-5108ebcc3ebf None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.292694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6beebad-be8a-40f3-8ae1-d3196043c759 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.307692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:07.328979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880378b4-6389-4aa6-8d50-b8d7ea70bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.329583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880378b4-6389-4aa6-8d50-b8d7ea70bce6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.329869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880378b4-6389-4aa6-8d50-b8d7ea70bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.330147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880378b4-6389-4aa6-8d50-b8d7ea70bce6 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.330685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:07.331444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880378b4-6389-4aa6-8d50-b8d7ea70bce6 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.333458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880378b4-6389-4aa6-8d50-b8d7ea70bce6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.335178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 47/86] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:07.335949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880378b4-6389-4aa6-8d50-b8d7ea70bce6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.374269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286f965-696c-417c-896b-5108ebcc3ebf None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.407632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062dbcbe-0f64-4ad4-a38e-77f7a5160690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.408250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062dbcbe-0f64-4ad4-a38e-77f7a5160690 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.408539 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062dbcbe-0f64-4ad4-a38e-77f7a5160690 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.408820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062dbcbe-0f64-4ad4-a38e-77f7a5160690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.437112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880378b4-6389-4aa6-8d50-b8d7ea70bce6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.474059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309440f-55ba-4983-bf54-02384a3fbb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.474744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309440f-55ba-4983-bf54-02384a3fbb54 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.475025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309440f-55ba-4983-bf54-02384a3fbb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.475299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309440f-55ba-4983-bf54-02384a3fbb54 None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.476470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309440f-55ba-4983-bf54-02384a3fbb54 None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.477966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309440f-55ba-4983-bf54-02384a3fbb54 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.479543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 48/87] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:07.480371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309440f-55ba-4983-bf54-02384a3fbb54 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.481710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062dbcbe-0f64-4ad4-a38e-77f7a5160690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.483204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062dbcbe-0f64-4ad4-a38e-77f7a5160690 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.528285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:07.546280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:07.579653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309440f-55ba-4983-bf54-02384a3fbb54 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.585751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 40/88] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:07.586569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062dbcbe-0f64-4ad4-a38e-77f7a5160690 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.612943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6441cd-614b-4354-b686-0cb48c3c4c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.614146 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6441cd-614b-4354-b686-0cb48c3c4c4a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.614434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6441cd-614b-4354-b686-0cb48c3c4c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.614712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6441cd-614b-4354-b686-0cb48c3c4c4a None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.615887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6441cd-614b-4354-b686-0cb48c3c4c4a None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.617346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6441cd-614b-4354-b686-0cb48c3c4c4a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.619010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 49/89] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:07.619770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6441cd-614b-4354-b686-0cb48c3c4c4a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.686183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062dbcbe-0f64-4ad4-a38e-77f7a5160690 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.718376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6441cd-614b-4354-b686-0cb48c3c4c4a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.719644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a0661-1304-4f2c-aace-8b4032150ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.720263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a0661-1304-4f2c-aace-8b4032150ac0 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.720556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a0661-1304-4f2c-aace-8b4032150ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.720829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a0661-1304-4f2c-aace-8b4032150ac0 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.721546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a0661-1304-4f2c-aace-8b4032150ac0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.723502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a0661-1304-4f2c-aace-8b4032150ac0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.735295 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:07.750402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 41/90] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:07 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:07.751252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a0661-1304-4f2c-aace-8b4032150ac0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.753794 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.754986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.755278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.755560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.756376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] RBAC: Adding query filter params (role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed, user.id=49483f572c9c483b9311fe1b812f3c38) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:07.756692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.758202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.764750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 50/91] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/role_assignments?user.id=49483f572c9c483b9311fe1b812f3c38&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=551979dba2834cf397f2c386c2c64b2d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:07.765730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:07.788097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:07.895042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a0661-1304-4f2c-aace-8b4032150ac0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.899013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a172a6-eaea-4bca-aac3-7b256390f1a3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:07.942414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811c68-0d62-4657-ae63-3dc7f952c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.943052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811c68-0d62-4657-ae63-3dc7f952c1ea None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.943334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811c68-0d62-4657-ae63-3dc7f952c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.943624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811c68-0d62-4657-ae63-3dc7f952c1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.946548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dbaf5-4b0f-42f8-9cee-58384f4f69e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:07.947440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbaf5-4b0f-42f8-9cee-58384f4f69e3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:07.948014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbaf5-4b0f-42f8-9cee-58384f4f69e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:07.948499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dbaf5-4b0f-42f8-9cee-58384f4f69e3 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:07.950209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dbaf5-4b0f-42f8-9cee-58384f4f69e3 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:07.951883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dbaf5-4b0f-42f8-9cee-58384f4f69e3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:07.954029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 51/92] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:07.955822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dbaf5-4b0f-42f8-9cee-58384f4f69e3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:08.028706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:08.038599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6811c68-0d62-4657-ae63-3dc7f952c1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:08.040401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6811c68-0d62-4657-ae63-3dc7f952c1ea None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:08.070158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:08.098871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dbaf5-4b0f-42f8-9cee-58384f4f69e3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:08.149985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:08.150954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:08.151444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:08.151917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:08.152919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed, user.id=5666224e1bf547bc87b970165a115158) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:08.153470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:08.155154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:08.163193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 52/93] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/role_assignments?user.id=5666224e1bf547bc87b970165a115158&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:08.163772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:08.184508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 42/94] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:08.192285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811c68-0d62-4657-ae63-3dc7f952c1ea None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:08.380574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811c68-0d62-4657-ae63-3dc7f952c1ea None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:08.395606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef4f0a-ea87-49d8-8eab-ed5e99f50db2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:08.404610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:08.421476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:08.424394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c737ce-29b6-4bb7-a4f0-cd245fc20ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:08.425211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c737ce-29b6-4bb7-a4f0-cd245fc20ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:08.425678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c737ce-29b6-4bb7-a4f0-cd245fc20ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:08.426398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c737ce-29b6-4bb7-a4f0-cd245fc20ce7 None admin] PATH_INFO: `/v3/projects/b0424e1775c04de0a035654307b41a02` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:08.428794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c737ce-29b6-4bb7-a4f0-cd245fc20ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=b0424e1775c04de0a035654307b41a02)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:08.430496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c737ce-29b6-4bb7-a4f0-cd245fc20ce7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:08.432926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 43/95] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:08 2022] GET /identity/v3/projects/b0424e1775c04de0a035654307b41a02 => generated 312 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:08.442986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c737ce-29b6-4bb7-a4f0-cd245fc20ce7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:08.468276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0d45a-6da9-4184-941c-65fa0efe3672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:08.469115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d45a-6da9-4184-941c-65fa0efe3672 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:08.469528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d45a-6da9-4184-941c-65fa0efe3672 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:08.469917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0d45a-6da9-4184-941c-65fa0efe3672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:08.470821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0d45a-6da9-4184-941c-65fa0efe3672 None admin] RBAC: Adding query filter params (role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed, user.id=49483f572c9c483b9311fe1b812f3c38) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:08.471240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0d45a-6da9-4184-941c-65fa0efe3672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:08.472891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0d45a-6da9-4184-941c-65fa0efe3672 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:08.483778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 53/96] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:08 2022] GET /identity/v3/role_assignments?user.id=49483f572c9c483b9311fe1b812f3c38&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=74f371799fd245bf987e798f5c342e6a => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:08.595550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c737ce-29b6-4bb7-a4f0-cd245fc20ce7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:08.645019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c849b1e-09b2-4cea-8f9f-c8ecfaf5061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:08.645699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849b1e-09b2-4cea-8f9f-c8ecfaf5061f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:08.645994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849b1e-09b2-4cea-8f9f-c8ecfaf5061f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:08.646274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849b1e-09b2-4cea-8f9f-c8ecfaf5061f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:08.647088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c849b1e-09b2-4cea-8f9f-c8ecfaf5061f None admin] RBAC: Adding query filter params (role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=b0424e1775c04de0a035654307b41a02, user.id=49483f572c9c483b9311fe1b812f3c38) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:08.647405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c849b1e-09b2-4cea-8f9f-c8ecfaf5061f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:08.648877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c849b1e-09b2-4cea-8f9f-c8ecfaf5061f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:08.657701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 44/97] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:08 2022] GET /identity/v3/role_assignments?user.id=49483f572c9c483b9311fe1b812f3c38&scope.project.id=b0424e1775c04de0a035654307b41a02&role.id=551979dba2834cf397f2c386c2c64b2d => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:10.283414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339660d5-9a9c-4462-a579-2a42afb5a2d5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:10.283925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339660d5-9a9c-4462-a579-2a42afb5a2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:10.283999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339660d5-9a9c-4462-a579-2a42afb5a2d5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:10.286129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 54/98] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:10.291530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df530c22-c6fb-4f63-8df7-3ccff409c330 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:10.291932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df530c22-c6fb-4f63-8df7-3ccff409c330 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:10.292097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df530c22-c6fb-4f63-8df7-3ccff409c330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:10.313734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-df530c22-c6fb-4f63-8df7-3ccff409c330 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:10.314945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-df530c22-c6fb-4f63-8df7-3ccff409c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:10.386088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:10.414764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 45/99] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:10.448090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339660d5-9a9c-4462-a579-2a42afb5a2d5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:10.595939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339660d5-9a9c-4462-a579-2a42afb5a2d5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:10.647500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4834ca-421d-4aa5-b50b-052944bc8059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:10.648165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4834ca-421d-4aa5-b50b-052944bc8059 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:10.648453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4834ca-421d-4aa5-b50b-052944bc8059 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:10.648733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4834ca-421d-4aa5-b50b-052944bc8059 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:10.662124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4834ca-421d-4aa5-b50b-052944bc8059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ce4834ca-421d-4aa5-b50b-052944bc8059 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:10.667157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:10.668004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:10.668765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 55/100] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:10.676066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df530c22-c6fb-4f63-8df7-3ccff409c330 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:10.802128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df530c22-c6fb-4f63-8df7-3ccff409c330 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:10.854987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:10.855546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:10.855832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:10.856110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:10.856840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:10.857153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:10.858572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:10.867706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2825-6249-4e06-96c7-ee053e317703 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:10.868000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2825-6249-4e06-96c7-ee053e317703 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:10.868279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2825-6249-4e06-96c7-ee053e317703 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:10.869927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 56/101] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:10.874700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d343-7482-4fb8-a29e-47232ed1b35d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:10.874995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d343-7482-4fb8-a29e-47232ed1b35d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:10.875278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d343-7482-4fb8-a29e-47232ed1b35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:10.890090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 46/102] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:10.895550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0fe4d343-7482-4fb8-a29e-47232ed1b35d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:10.896433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:10.906034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0fe4d343-7482-4fb8-a29e-47232ed1b35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:10.966720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:10.966913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:11.088081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 57/103] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:11.105987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72610c-0cf3-49ba-94cb-4844cfb86bb4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:11.134781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d343-7482-4fb8-a29e-47232ed1b35d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:11.157482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada18016-0d1b-46da-90eb-391d3b72598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:11.158482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada18016-0d1b-46da-90eb-391d3b72598b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.158973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada18016-0d1b-46da-90eb-391d3b72598b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.159453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada18016-0d1b-46da-90eb-391d3b72598b None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.161630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada18016-0d1b-46da-90eb-391d3b72598b None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:11.163317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada18016-0d1b-46da-90eb-391d3b72598b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:11.165318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 47/104] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:10 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:11.166799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355c655-8ff1-4ec5-a867-f2a8372a9a27 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.167295 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355c655-8ff1-4ec5-a867-f2a8372a9a27 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.167798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355c655-8ff1-4ec5-a867-f2a8372a9a27 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.169424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 48/105] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:11.171749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355c655-8ff1-4ec5-a867-f2a8372a9a27 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:11.302977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355c655-8ff1-4ec5-a867-f2a8372a9a27 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:11.323498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:11.340425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d343-7482-4fb8-a29e-47232ed1b35d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:11.356150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa77c5c-9da7-4e2b-9366-fed4b378d289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:11.356977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77c5c-9da7-4e2b-9366-fed4b378d289 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.357265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77c5c-9da7-4e2b-9366-fed4b378d289 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.357574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77c5c-9da7-4e2b-9366-fed4b378d289 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.359210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa77c5c-9da7-4e2b-9366-fed4b378d289 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:11.360942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa77c5c-9da7-4e2b-9366-fed4b378d289 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:11.362808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 49/106] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:11 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:11.365308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafad2e-443c-45f9-99f3-e3f2a785f35b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.365848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafad2e-443c-45f9-99f3-e3f2a785f35b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.366333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafad2e-443c-45f9-99f3-e3f2a785f35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.387577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344dce73-7257-4bd7-a429-4a82f33ecbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:11.388633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dce73-7257-4bd7-a429-4a82f33ecbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.389131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dce73-7257-4bd7-a429-4a82f33ecbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.389672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344dce73-7257-4bd7-a429-4a82f33ecbe8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.391054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fcafad2e-443c-45f9-99f3-e3f2a785f35b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:11.392332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fcafad2e-443c-45f9-99f3-e3f2a785f35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:11.403378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344dce73-7257-4bd7-a429-4a82f33ecbe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-344dce73-7257-4bd7-a429-4a82f33ecbe8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:11.408898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:11.409861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:11.411166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:11.413973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 58/107] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:11.415450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33492fb9-35b8-426c-8f68-aceaea2c2c8c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.415951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33492fb9-35b8-426c-8f68-aceaea2c2c8c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.416437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33492fb9-35b8-426c-8f68-aceaea2c2c8c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.418232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 59/108] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:11.423137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33492fb9-35b8-426c-8f68-aceaea2c2c8c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:11.454148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:11.481467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 50/109] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:11.490456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafad2e-443c-45f9-99f3-e3f2a785f35b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:11.547610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33492fb9-35b8-426c-8f68-aceaea2c2c8c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:11.584415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43b9a9-41c9-4086-8cf3-dc0144dc9b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:11.585488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b9a9-41c9-4086-8cf3-dc0144dc9b63 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.585979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b9a9-41c9-4086-8cf3-dc0144dc9b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.586451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b9a9-41c9-4086-8cf3-dc0144dc9b63 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/users/49483f572c9c483b9311fe1b812f3c38/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.588664 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43b9a9-41c9-4086-8cf3-dc0144dc9b63 None admin] RBAC: Authorizing `identity:create_grant(project_id=89d1fa86df9a40fe96f2a7554edbbaed, user_id=49483f572c9c483b9311fe1b812f3c38, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:11.590446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43b9a9-41c9-4086-8cf3-dc0144dc9b63 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:11.607943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 60/110] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:11 2022] PUT /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/users/49483f572c9c483b9311fe1b812f3c38/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:11.609670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30346-2a2e-4b27-ac07-6356a3ccc0a1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.610171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30346-2a2e-4b27-ac07-6356a3ccc0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.610896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30346-2a2e-4b27-ac07-6356a3ccc0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.619693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafad2e-443c-45f9-99f3-e3f2a785f35b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:11.632761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0ea30346-2a2e-4b27-ac07-6356a3ccc0a1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:11.634549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0ea30346-2a2e-4b27-ac07-6356a3ccc0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:11.643717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:11.697686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:11.699213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.699607 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.699945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.701020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:11.701362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:11.703193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:11.717330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 51/111] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:11.718569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:11.724692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 61/112] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:11.726392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0aa7-fda7-4fb7-b680-746c20079583 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.726901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0aa7-fda7-4fb7-b680-746c20079583 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.727411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba0aa7-fda7-4fb7-b680-746c20079583 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.729218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 62/113] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:11.730211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba0aa7-fda7-4fb7-b680-746c20079583 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:11.769406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:11.860206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b200-8c63-4ae0-a159-d3e2e5108f08 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:11.871044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba0aa7-fda7-4fb7-b680-746c20079583 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:11.896919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a7e12-ed0a-4b6b-8682-0ec00cabe110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:11.897595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7e12-ed0a-4b6b-8682-0ec00cabe110 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.897879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7e12-ed0a-4b6b-8682-0ec00cabe110 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.898157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7e12-ed0a-4b6b-8682-0ec00cabe110 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.905859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2818350-b4e2-4148-b42c-ca1d6e6462e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:11.906497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2818350-b4e2-4148-b42c-ca1d6e6462e0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.906779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2818350-b4e2-4148-b42c-ca1d6e6462e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.907075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2818350-b4e2-4148-b42c-ca1d6e6462e0 None admin] PATH_INFO: `/v3/users/5666224e1bf547bc87b970165a115158` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.908380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2818350-b4e2-4148-b42c-ca1d6e6462e0 None admin] RBAC: Authorizing `identity:get_user(user_id=5666224e1bf547bc87b970165a115158)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:11.908745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a7e12-ed0a-4b6b-8682-0ec00cabe110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:11.909847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2818350-b4e2-4148-b42c-ca1d6e6462e0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:11.912244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 63/114] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:11 2022] GET /identity/v3/users/5666224e1bf547bc87b970165a115158 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:11.913188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64f25e-42ca-40a0-8af9-ed3a6a7c41cd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:11.913494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64f25e-42ca-40a0-8af9-ed3a6a7c41cd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:11.913770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64f25e-42ca-40a0-8af9-ed3a6a7c41cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a39a7e12-ed0a-4b6b-8682-0ec00cabe110 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:11.919252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:11.920117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:11.920888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 52/115] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:11.921606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a7e12-ed0a-4b6b-8682-0ec00cabe110 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:11.937476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9b64f25e-42ca-40a0-8af9-ed3a6a7c41cd None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:11.938818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b64f25e-42ca-40a0-8af9-ed3a6a7c41cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:11.954753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:12.012388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 64/116] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:12.013510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e177-2559-42b1-a74d-b4fcec5ea786 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.013800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e177-2559-42b1-a74d-b4fcec5ea786 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.014076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e177-2559-42b1-a74d-b4fcec5ea786 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.015314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 65/117] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:12.016028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e177-2559-42b1-a74d-b4fcec5ea786 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.038726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a7e12-ed0a-4b6b-8682-0ec00cabe110 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.045640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:12.077803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cfdf5-c1d4-4046-82af-899b1ab1057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.078725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cfdf5-c1d4-4046-82af-899b1ab1057f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.079234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cfdf5-c1d4-4046-82af-899b1ab1057f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.079517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cfdf5-c1d4-4046-82af-899b1ab1057f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.080682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cfdf5-c1d4-4046-82af-899b1ab1057f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:12.082226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cfdf5-c1d4-4046-82af-899b1ab1057f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:12.083845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 53/118] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:12.084609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cfdf5-c1d4-4046-82af-899b1ab1057f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.142787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e177-2559-42b1-a74d-b4fcec5ea786 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.180360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ac98-d55e-458d-b630-b08f65e84da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.181009 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ac98-d55e-458d-b630-b08f65e84da5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.181321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ac98-d55e-458d-b630-b08f65e84da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.181600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ac98-d55e-458d-b630-b08f65e84da5 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.182766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385ac98-d55e-458d-b630-b08f65e84da5 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:12.184202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385ac98-d55e-458d-b630-b08f65e84da5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:12.186047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 66/119] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:12 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:12.186996 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8095c2-7bf7-47a0-a452-ceb77dbd621e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.187279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8095c2-7bf7-47a0-a452-ceb77dbd621e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.187558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8095c2-7bf7-47a0-a452-ceb77dbd621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.196818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:12.200884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cfdf5-c1d4-4046-82af-899b1ab1057f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.205540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5c8095c2-7bf7-47a0-a452-ceb77dbd621e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:12.206717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5c8095c2-7bf7-47a0-a452-ceb77dbd621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:12.238708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.239337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.239624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.239899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.240610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:12.240938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:12.242330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:12.251431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 54/120] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:12.254653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.279026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 67/121] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:12.279862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8095c2-7bf7-47a0-a452-ceb77dbd621e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.302514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:12.382246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1dbc5-e6ee-456f-848e-9bafa5cabcf6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.387163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8095c2-7bf7-47a0-a452-ceb77dbd621e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.420934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564acdd-c5a7-42e0-9fd7-3c7673802394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.421836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564acdd-c5a7-42e0-9fd7-3c7673802394 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.422275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564acdd-c5a7-42e0-9fd7-3c7673802394 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.422873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564acdd-c5a7-42e0-9fd7-3c7673802394 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.425464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112d59-a80e-4578-aa42-321bd58a5d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.426627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112d59-a80e-4578-aa42-321bd58a5d73 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.426916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112d59-a80e-4578-aa42-321bd58a5d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.427195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112d59-a80e-4578-aa42-321bd58a5d73 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.427890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81112d59-a80e-4578-aa42-321bd58a5d73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:12.429382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81112d59-a80e-4578-aa42-321bd58a5d73 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:12.438034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564acdd-c5a7-42e0-9fd7-3c7673802394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b564acdd-c5a7-42e0-9fd7-3c7673802394 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:12.444041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:12.445083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:12.446044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:12.448204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 55/122] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:12.449479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564acdd-c5a7-42e0-9fd7-3c7673802394 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.473321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 68/123] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:23:12 2022] POST /identity/v3/projects => generated 302 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:12.473321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112d59-a80e-4578-aa42-321bd58a5d73 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.492584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:12.580533 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:12.599243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564acdd-c5a7-42e0-9fd7-3c7673802394 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.657083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33120a-5ad1-4865-8271-ee4332d0f1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.657724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112d59-a80e-4578-aa42-321bd58a5d73 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.657831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33120a-5ad1-4865-8271-ee4332d0f1fb None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.658082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33120a-5ad1-4865-8271-ee4332d0f1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.658412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33120a-5ad1-4865-8271-ee4332d0f1fb None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/users/5666224e1bf547bc87b970165a115158/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.661963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33120a-5ad1-4865-8271-ee4332d0f1fb None admin] RBAC: Authorizing `identity:create_grant(project_id=89d1fa86df9a40fe96f2a7554edbbaed, user_id=5666224e1bf547bc87b970165a115158, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:12.664323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33120a-5ad1-4865-8271-ee4332d0f1fb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:12.685431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 56/124] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:12 2022] PUT /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/users/5666224e1bf547bc87b970165a115158/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:12.685431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33120a-5ad1-4865-8271-ee4332d0f1fb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.725211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863afe1-41d7-4584-8611-e31d200bec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.726111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863afe1-41d7-4584-8611-e31d200bec47 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.726395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863afe1-41d7-4584-8611-e31d200bec47 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.726776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863afe1-41d7-4584-8611-e31d200bec47 None admin] PATH_INFO: `/v3/users/722cc12185804086810585b612f1e991` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.743940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863afe1-41d7-4584-8611-e31d200bec47 None admin] RBAC: Authorizing `identity:get_user(user_id=722cc12185804086810585b612f1e991)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:12.745489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863afe1-41d7-4584-8611-e31d200bec47 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:12.747503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 69/125] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:12 2022] GET /identity/v3/users/722cc12185804086810585b612f1e991 => generated 289 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:12.748539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863afe1-41d7-4584-8611-e31d200bec47 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.809983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:12.835670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33120a-5ad1-4865-8271-ee4332d0f1fb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.887188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1d499-b3d1-487b-9fad-0e7aa584189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.888212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1d499-b3d1-487b-9fad-0e7aa584189b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.888753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1d499-b3d1-487b-9fad-0e7aa584189b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.889354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1d499-b3d1-487b-9fad-0e7aa584189b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.902451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1d499-b3d1-487b-9fad-0e7aa584189b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:12.906617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863afe1-41d7-4584-8611-e31d200bec47 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-4aa1d499-b3d1-487b-9fad-0e7aa584189b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:12.909286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:12.910739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:12.911516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:12.914232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 57/126] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:12.915285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1d499-b3d1-487b-9fad-0e7aa584189b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:12.949753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:12.966338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:12.967006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:12.967294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:12.967568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:12.968300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:12.969217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:12.970669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:12.991334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 70/127] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:12.993277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:13.101709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1d499-b3d1-487b-9fad-0e7aa584189b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:13.155945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b1396-467a-4a0f-a0a1-b12676b5b3f4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:13.178046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ebab1-d571-4439-aa2e-47a8be40b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:13.178696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ebab1-d571-4439-aa2e-47a8be40b726 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:13.178987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ebab1-d571-4439-aa2e-47a8be40b726 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:13.179274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ebab1-d571-4439-aa2e-47a8be40b726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:13.192469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:13.192736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:13.193538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:13.193892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:13.194234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:13.195029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:13.195349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:13.196796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:13.206121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 71/128] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:13.218345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:13.255821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ebab1-d571-4439-aa2e-47a8be40b726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:13.257343 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ebab1-d571-4439-aa2e-47a8be40b726 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:13.300999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:13.395201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaeb954-525d-428d-84b2-eae1a22aa839 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:13.404244 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 58/129] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:13.405329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ebab1-d571-4439-aa2e-47a8be40b726 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:13.453723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:13.477021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ca275-934d-4560-a4c1-aadf7af4f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:13.477720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca275-934d-4560-a4c1-aadf7af4f154 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:13.478013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca275-934d-4560-a4c1-aadf7af4f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:13.478556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca275-934d-4560-a4c1-aadf7af4f154 None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:13.479754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ca275-934d-4560-a4c1-aadf7af4f154 None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:13.481257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ca275-934d-4560-a4c1-aadf7af4f154 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:13.483480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 72/130] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:13 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:13.484139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ca275-934d-4560-a4c1-aadf7af4f154 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:13.562578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ebab1-d571-4439-aa2e-47a8be40b726 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:13.635818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec928c63-8f77-47a4-8667-aa014ffabb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:13.636476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec928c63-8f77-47a4-8667-aa014ffabb07 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:13.636764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec928c63-8f77-47a4-8667-aa014ffabb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:13.637046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec928c63-8f77-47a4-8667-aa014ffabb07 None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:13.638991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec928c63-8f77-47a4-8667-aa014ffabb07 None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:13.640459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec928c63-8f77-47a4-8667-aa014ffabb07 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:13.645883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 59/131] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:13 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:13.646699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec928c63-8f77-47a4-8667-aa014ffabb07 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:13.684313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ca275-934d-4560-a4c1-aadf7af4f154 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:13.717672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:13.749618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03da82-054a-4ed7-a189-98497526a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:13.750192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03da82-054a-4ed7-a189-98497526a9da None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:13.750480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03da82-054a-4ed7-a189-98497526a9da None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:13.750761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03da82-054a-4ed7-a189-98497526a9da None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:13.753048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03da82-054a-4ed7-a189-98497526a9da None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:13.755308 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03da82-054a-4ed7-a189-98497526a9da None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:13.757161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 73/132] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:13 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:13.757794 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03da82-054a-4ed7-a189-98497526a9da None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:13.835258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec928c63-8f77-47a4-8667-aa014ffabb07 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:13.853032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:13.899313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f57dc-4a95-4a4d-a896-c03920b9359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:13.900349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f57dc-4a95-4a4d-a896-c03920b9359e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:13.900838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f57dc-4a95-4a4d-a896-c03920b9359e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:13.901335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f57dc-4a95-4a4d-a896-c03920b9359e None admin] PATH_INFO: `/v3/projects/b0424e1775c04de0a035654307b41a02` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:13.904218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f57dc-4a95-4a4d-a896-c03920b9359e None admin] RBAC: Authorizing `identity:get_project(project_id=b0424e1775c04de0a035654307b41a02)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:13.905933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f57dc-4a95-4a4d-a896-c03920b9359e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:13.907843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 60/133] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:13 2022] GET /identity/v3/projects/b0424e1775c04de0a035654307b41a02 => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:13.910647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f57dc-4a95-4a4d-a896-c03920b9359e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:13.953226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03da82-054a-4ed7-a189-98497526a9da None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:14.009986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e88f79-1b77-48e5-b8fc-3ae56d8529f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:14.010559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e88f79-1b77-48e5-b8fc-3ae56d8529f4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:14.010850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e88f79-1b77-48e5-b8fc-3ae56d8529f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:14.011128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e88f79-1b77-48e5-b8fc-3ae56d8529f4 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:14.012456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e88f79-1b77-48e5-b8fc-3ae56d8529f4 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:14.014580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e88f79-1b77-48e5-b8fc-3ae56d8529f4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:14.017183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 74/134] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:13 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:14.023316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e88f79-1b77-48e5-b8fc-3ae56d8529f4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:14.071032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f57dc-4a95-4a4d-a896-c03920b9359e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:14.092315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:14.119777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e957d8-4040-468c-be23-a45dfeedb5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:14.120851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e957d8-4040-468c-be23-a45dfeedb5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:14.122132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e957d8-4040-468c-be23-a45dfeedb5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:14.122641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e957d8-4040-468c-be23-a45dfeedb5c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:14.123637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e957d8-4040-468c-be23-a45dfeedb5c6 None admin] RBAC: Adding query filter params (role.id=3cc2fe72d0a245238f0d8df6763755c9, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed, user.id=722cc12185804086810585b612f1e991) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:14.124152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e957d8-4040-468c-be23-a45dfeedb5c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:14.125856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e957d8-4040-468c-be23-a45dfeedb5c6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:14.133396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 61/135] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:13 2022] GET /identity/v3/role_assignments?user.id=722cc12185804086810585b612f1e991&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=3cc2fe72d0a245238f0d8df6763755c9 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:14.135365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4203-f415-403a-b5b5-623524f1f007 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:14.136141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4203-f415-403a-b5b5-623524f1f007 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:14.136635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4203-f415-403a-b5b5-623524f1f007 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:14.138718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 62/136] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:14.139746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc4203-f415-403a-b5b5-623524f1f007 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:14.161014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e88f79-1b77-48e5-b8fc-3ae56d8529f4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:14.209674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:14.230316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7e736-5197-4aec-813e-7000b8d0a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:14.231392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7e736-5197-4aec-813e-7000b8d0a30b None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:14.231914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7e736-5197-4aec-813e-7000b8d0a30b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:14.232396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7e736-5197-4aec-813e-7000b8d0a30b None admin] PATH_INFO: `/v3/projects/b0424e1775c04de0a035654307b41a02/users/49483f572c9c483b9311fe1b812f3c38/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:14.236048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7e736-5197-4aec-813e-7000b8d0a30b None admin] RBAC: Authorizing `identity:create_grant(project_id=b0424e1775c04de0a035654307b41a02, user_id=49483f572c9c483b9311fe1b812f3c38, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:14.239309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7e736-5197-4aec-813e-7000b8d0a30b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:14.263345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 75/137] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:14 2022] PUT /identity/v3/projects/b0424e1775c04de0a035654307b41a02/users/49483f572c9c483b9311fe1b812f3c38/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:14.266162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5208b9-737d-49c5-b0ab-3f37c058511f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:14.267894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5208b9-737d-49c5-b0ab-3f37c058511f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:14.268558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5208b9-737d-49c5-b0ab-3f37c058511f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:14.305362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ef5208b9-737d-49c5-b0ab-3f37c058511f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:14.307413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ef5208b9-737d-49c5-b0ab-3f37c058511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:14.340743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc4203-f415-403a-b5b5-623524f1f007 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:14.383147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe494d-c015-40c4-a991-cd66d7e1d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:14.383785 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe494d-c015-40c4-a991-cd66d7e1d1e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:14.384078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe494d-c015-40c4-a991-cd66d7e1d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:14.384359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe494d-c015-40c4-a991-cd66d7e1d1e5 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/users/49483f572c9c483b9311fe1b812f3c38/roles/74f371799fd245bf987e798f5c342e6a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:14.387648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe494d-c015-40c4-a991-cd66d7e1d1e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=89d1fa86df9a40fe96f2a7554edbbaed, user_id=49483f572c9c483b9311fe1b812f3c38, role_id=74f371799fd245bf987e798f5c342e6a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:14.390291 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe494d-c015-40c4-a991-cd66d7e1d1e5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:14.399906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 76/138] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:14.418960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 63/139] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:14 2022] PUT /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/users/49483f572c9c483b9311fe1b812f3c38/roles/74f371799fd245bf987e798f5c342e6a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:14.462043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5208b9-737d-49c5-b0ab-3f37c058511f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:14.537901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:14.622194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5208b9-737d-49c5-b0ab-3f37c058511f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:14.666986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae39219-e958-496c-b768-a1c78e87402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:14.667657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae39219-e958-496c-b768-a1c78e87402b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:14.667942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae39219-e958-496c-b768-a1c78e87402b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:14.668296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae39219-e958-496c-b768-a1c78e87402b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:14.680040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae39219-e958-496c-b768-a1c78e87402b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-fae39219-e958-496c-b768-a1c78e87402b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:14.685104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:14.685989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:14.687018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 77/140] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:14.695531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe494d-c015-40c4-a991-cd66d7e1d1e5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:14.824681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe494d-c015-40c4-a991-cd66d7e1d1e5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:14.826554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:14.873227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:14.873882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:14.874166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:14.874443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:14.875162 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:14.875464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:14.876835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:14.888780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 64/141] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:14.895338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae39219-e958-496c-b768-a1c78e87402b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:15.046516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae39219-e958-496c-b768-a1c78e87402b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:15.048088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:15.089770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d46dc-3184-4704-ac8d-fe4c0183cdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:15.090331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d46dc-3184-4704-ac8d-fe4c0183cdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.090612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d46dc-3184-4704-ac8d-fe4c0183cdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.090884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d46dc-3184-4704-ac8d-fe4c0183cdc7 None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.094061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d46dc-3184-4704-ac8d-fe4c0183cdc7 None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:15.095880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d46dc-3184-4704-ac8d-fe4c0183cdc7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:15.098552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 78/142] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:14 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:15.105000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:15.254535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c1f6-0429-4970-9614-618d57a66f13 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.255230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c1f6-0429-4970-9614-618d57a66f13 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.255720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c1f6-0429-4970-9614-618d57a66f13 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.257324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 79/143] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:15.261041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546bf4-7fc9-4a9f-b8ed-070566d44972 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:15.262620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903a3c-29cd-4191-bf6f-ca6efe200276 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.263113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903a3c-29cd-4191-bf6f-ca6efe200276 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.263586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903a3c-29cd-4191-bf6f-ca6efe200276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.290170 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-56903a3c-29cd-4191-bf6f-ca6efe200276 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:15.294132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-56903a3c-29cd-4191-bf6f-ca6efe200276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:15.318655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c5e4f-200c-437d-b879-1c3dc108ec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:15.319397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c5e4f-200c-437d-b879-1c3dc108ec2d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.319691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c5e4f-200c-437d-b879-1c3dc108ec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.319969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c5e4f-200c-437d-b879-1c3dc108ec2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.355119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:15.407518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c5e4f-200c-437d-b879-1c3dc108ec2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:15.410162 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c5e4f-200c-437d-b879-1c3dc108ec2d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:15.479616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 80/144] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:15.481727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab577ad-f4fc-4534-8d06-ccd0df0e5bc8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.482234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab577ad-f4fc-4534-8d06-ccd0df0e5bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.482704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab577ad-f4fc-4534-8d06-ccd0df0e5bc8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.484149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 81/145] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:15.494562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee1bc-1f3b-4815-8e1e-2fc80a397375 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.495122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee1bc-1f3b-4815-8e1e-2fc80a397375 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.495594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee1bc-1f3b-4815-8e1e-2fc80a397375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.522329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-31dee1bc-1f3b-4815-8e1e-2fc80a397375 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:15.529837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-31dee1bc-1f3b-4815-8e1e-2fc80a397375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:15.549240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 65/146] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:15.551182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c5e4f-200c-437d-b879-1c3dc108ec2d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:15.620973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:15.664560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c5e4f-200c-437d-b879-1c3dc108ec2d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:15.686495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 82/147] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:15.688413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee1bc-1f3b-4815-8e1e-2fc80a397375 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:15.702263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd5be2-ecad-4f21-897f-a3294e87433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:15.703242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd5be2-ecad-4f21-897f-a3294e87433f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.703768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd5be2-ecad-4f21-897f-a3294e87433f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.704299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd5be2-ecad-4f21-897f-a3294e87433f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.720065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd5be2-ecad-4f21-897f-a3294e87433f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-dbdd5be2-ecad-4f21-897f-a3294e87433f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:15.726503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:15.727698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:15.728934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:15.732506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 66/148] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:15.741756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd5be2-ecad-4f21-897f-a3294e87433f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:15.742915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:15.888338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd5be2-ecad-4f21-897f-a3294e87433f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:15.915321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:15.915489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee1bc-1f3b-4815-8e1e-2fc80a397375 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:15.928964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:15.929923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.930415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.930910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.931849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:15.932370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:15.933972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:15.944319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 67/149] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:15.945571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:15.997458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c62696-df19-440f-ab79-d8b7931d0d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:15.998115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c62696-df19-440f-ab79-d8b7931d0d3f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:15.998397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c62696-df19-440f-ab79-d8b7931d0d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:15.998669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c62696-df19-440f-ab79-d8b7931d0d3f None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:15.999834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c62696-df19-440f-ab79-d8b7931d0d3f None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:16.001278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c62696-df19-440f-ab79-d8b7931d0d3f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:16.002961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 83/150] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:15 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:16.008802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c62696-df19-440f-ab79-d8b7931d0d3f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:16.058991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0a6ed-f48a-4e56-98e2-f5d11654505f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:16.107671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed76927-5ea2-4253-989b-ff6dacea87c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:16.109247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed76927-5ea2-4253-989b-ff6dacea87c9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:16.110315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed76927-5ea2-4253-989b-ff6dacea87c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:16.111046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed76927-5ea2-4253-989b-ff6dacea87c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:16.118791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed76927-5ea2-4253-989b-ff6dacea87c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:16.122560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed76927-5ea2-4253-989b-ff6dacea87c9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:16.125231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 68/151] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:16.127818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed76927-5ea2-4253-989b-ff6dacea87c9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:16.194112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:16.200030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c62696-df19-440f-ab79-d8b7931d0d3f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:16.241104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fc509-03d0-48b7-9b66-e5c855bf3b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:16.242165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc509-03d0-48b7-9b66-e5c855bf3b1f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:16.242679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc509-03d0-48b7-9b66-e5c855bf3b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:16.243156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc509-03d0-48b7-9b66-e5c855bf3b1f None admin] PATH_INFO: `/v3/users/5666224e1bf547bc87b970165a115158` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:16.245128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fc509-03d0-48b7-9b66-e5c855bf3b1f None admin] RBAC: Authorizing `identity:get_user(user_id=5666224e1bf547bc87b970165a115158)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:16.246933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fc509-03d0-48b7-9b66-e5c855bf3b1f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:16.249098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 84/152] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:16 2022] GET /identity/v3/users/5666224e1bf547bc87b970165a115158 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:16.250073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fc509-03d0-48b7-9b66-e5c855bf3b1f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:16.288264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:16.374135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fc509-03d0-48b7-9b66-e5c855bf3b1f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:16.414599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ecf5a-278d-460c-92d9-a58348711ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:16.415671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ecf5a-278d-460c-92d9-a58348711ee3 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:16.416314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ecf5a-278d-460c-92d9-a58348711ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:16.417422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ecf5a-278d-460c-92d9-a58348711ee3 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:16.419861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed76927-5ea2-4253-989b-ff6dacea87c9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:16.420556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ecf5a-278d-460c-92d9-a58348711ee3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:16.422243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ecf5a-278d-460c-92d9-a58348711ee3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:16.476968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 85/153] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:16 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:16.479481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ecf5a-278d-460c-92d9-a58348711ee3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:16.526278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:16.527348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:16.528016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:16.529588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:16.530335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:16.533267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] RBAC: Adding query filter params (role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed, user.id=49483f572c9c483b9311fe1b812f3c38) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:16.534051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:16.535958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:16.544839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 69/154] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:16 2022] GET /identity/v3/role_assignments?user.id=49483f572c9c483b9311fe1b812f3c38&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=551979dba2834cf397f2c386c2c64b2d => generated 608 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:16.644249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ecf5a-278d-460c-92d9-a58348711ee3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:16.682831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ed8ff-6b18-4d04-af95-c421e7c169a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:16.683481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ed8ff-6b18-4d04-af95-c421e7c169a3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:16.683765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ed8ff-6b18-4d04-af95-c421e7c169a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:16.684041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ed8ff-6b18-4d04-af95-c421e7c169a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:16.760481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ed8ff-6b18-4d04-af95-c421e7c169a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:16.762034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ed8ff-6b18-4d04-af95-c421e7c169a3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:16.788656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:16.873609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 86/155] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:16.876022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:16.970598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c32e8-6f9b-4427-9097-722465d99687 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:16.970920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c32e8-6f9b-4427-9097-722465d99687 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:16.971159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c32e8-6f9b-4427-9097-722465d99687 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:16.972634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 87/156] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:16.978320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba97ee6-e2e0-4352-b882-f267deb6a753 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:16.978601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba97ee6-e2e0-4352-b882-f267deb6a753 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:16.978878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba97ee6-e2e0-4352-b882-f267deb6a753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.006673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13390e-21bb-42c2-8926-4f61f9eb1d98 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:17.011031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-eba97ee6-e2e0-4352-b882-f267deb6a753 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:17.012171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-eba97ee6-e2e0-4352-b882-f267deb6a753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:17.030187 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:17.058857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eae24-1a18-4b5d-a721-a5f363c21668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:17.059901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eae24-1a18-4b5d-a721-a5f363c21668 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.060282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eae24-1a18-4b5d-a721-a5f363c21668 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.060849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eae24-1a18-4b5d-a721-a5f363c21668 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.062078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8eae24-1a18-4b5d-a721-a5f363c21668 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:17.063653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8eae24-1a18-4b5d-a721-a5f363c21668 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:17.066974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 70/157] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:16 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:17.075234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eae24-1a18-4b5d-a721-a5f363c21668 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:17.113760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 88/158] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:17.146683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba97ee6-e2e0-4352-b882-f267deb6a753 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:17.211284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eae24-1a18-4b5d-a721-a5f363c21668 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:17.216870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:17.255730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8ad44-6a66-4e5b-868c-9670d0befc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:17.256366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8ad44-6a66-4e5b-868c-9670d0befc35 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.256654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8ad44-6a66-4e5b-868c-9670d0befc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.256933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8ad44-6a66-4e5b-868c-9670d0befc35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.257786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8ad44-6a66-4e5b-868c-9670d0befc35 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed, user.id=5666224e1bf547bc87b970165a115158) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:17.258100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8ad44-6a66-4e5b-868c-9670d0befc35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:17.259595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8ad44-6a66-4e5b-868c-9670d0befc35 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:17.267053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 71/159] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:17 2022] GET /identity/v3/role_assignments?user.id=5666224e1bf547bc87b970165a115158&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:17.268227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b704-6045-44bf-bc99-a9ea824e2b7a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.268515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b704-6045-44bf-bc99-a9ea824e2b7a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.268804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692b704-6045-44bf-bc99-a9ea824e2b7a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.271018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 72/160] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:17.277722 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938dff0-3bc6-420f-8226-a9db6d2e0b36 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.278011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938dff0-3bc6-420f-8226-a9db6d2e0b36 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.278278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938dff0-3bc6-420f-8226-a9db6d2e0b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.289821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba97ee6-e2e0-4352-b882-f267deb6a753 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:17.306747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9938dff0-3bc6-420f-8226-a9db6d2e0b36 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:17.308018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9938dff0-3bc6-420f-8226-a9db6d2e0b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:17.373874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:17.381509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415dde-42f2-4677-8ca8-482768a98274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:17.382181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415dde-42f2-4677-8ca8-482768a98274 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.384550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415dde-42f2-4677-8ca8-482768a98274 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.384840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415dde-42f2-4677-8ca8-482768a98274 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.410973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415dde-42f2-4677-8ca8-482768a98274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:17.414700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 73/161] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2c415dde-42f2-4677-8ca8-482768a98274 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:17.416816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:17.417697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:17.418914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:17.422549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 89/162] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:17.427913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62ddea-dd5b-4409-9453-a798802f854c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.428479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62ddea-dd5b-4409-9453-a798802f854c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.428953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62ddea-dd5b-4409-9453-a798802f854c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.430609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 74/163] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:17.436274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415dde-42f2-4677-8ca8-482768a98274 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:17.436920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b085-2981-4d33-a396-e94b02302f1f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.437542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b085-2981-4d33-a396-e94b02302f1f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.438020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b085-2981-4d33-a396-e94b02302f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.469550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f1a6b085-2981-4d33-a396-e94b02302f1f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:17.471058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f1a6b085-2981-4d33-a396-e94b02302f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:17.559283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 75/164] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:17.560259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6b085-2981-4d33-a396-e94b02302f1f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:17.564796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415dde-42f2-4677-8ca8-482768a98274 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:17.580663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:17.605711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:17.606619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.607049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.607469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.608356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:17.608871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:17.610534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:17.621200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 90/165] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:17.622718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:17.715916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6b085-2981-4d33-a396-e94b02302f1f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:17.732268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:17.753590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228ada7-8b2f-46af-9af1-c41bcc91e8cf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:17.769010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d37b2-7043-4fb6-a71d-4c8cba0348f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:17.770347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d37b2-7043-4fb6-a71d-4c8cba0348f8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.770759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d37b2-7043-4fb6-a71d-4c8cba0348f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.771139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d37b2-7043-4fb6-a71d-4c8cba0348f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.785768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d37b2-7043-4fb6-a71d-4c8cba0348f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-861d37b2-7043-4fb6-a71d-4c8cba0348f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:17.792151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:17.793123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:17.793943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:17.800473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 76/166] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:17.800473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d37b2-7043-4fb6-a71d-4c8cba0348f8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:17.822297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c674d8e-a2e4-4261-bcd1-081c1fe3b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:17.823342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c674d8e-a2e4-4261-bcd1-081c1fe3b476 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.823851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c674d8e-a2e4-4261-bcd1-081c1fe3b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.824680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c674d8e-a2e4-4261-bcd1-081c1fe3b476 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.838104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c674d8e-a2e4-4261-bcd1-081c1fe3b476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3c674d8e-a2e4-4261-bcd1-081c1fe3b476 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:17.845010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:17.846035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:17.846861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:17.850031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 91/167] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:17.851210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c674d8e-a2e4-4261-bcd1-081c1fe3b476 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:17.892994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:17.931094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d37b2-7043-4fb6-a71d-4c8cba0348f8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:17.973386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5cc27-cc5c-4f24-b7ff-07cb0d416573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:17.973386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5cc27-cc5c-4f24-b7ff-07cb0d416573 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:17.973386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5cc27-cc5c-4f24-b7ff-07cb0d416573 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:17.973386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5cc27-cc5c-4f24-b7ff-07cb0d416573 None admin] PATH_INFO: `/v3/users/722cc12185804086810585b612f1e991` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:17.978315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5cc27-cc5c-4f24-b7ff-07cb0d416573 None admin] RBAC: Authorizing `identity:get_user(user_id=722cc12185804086810585b612f1e991)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:17.978315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5cc27-cc5c-4f24-b7ff-07cb0d416573 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:17.979712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 77/168] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:17 2022] GET /identity/v3/users/722cc12185804086810585b612f1e991 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:17.980309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5cc27-cc5c-4f24-b7ff-07cb0d416573 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:17.989979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c674d8e-a2e4-4261-bcd1-081c1fe3b476 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:18.055758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:18.056419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:18.056709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:18.056994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:18.057765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:18.058076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:18.059488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:18.060763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:18.084549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 92/169] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:18.087266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:18.171380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5cc27-cc5c-4f24-b7ff-07cb0d416573 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:18.217910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:18.218542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:18.218822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:18.219094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:18.219817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:18.220127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:18.222313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:18.239449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 78/170] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:18.240966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:18.261939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:18.271277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e33a2-830c-4eeb-bee0-b66189543467 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:18.338970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b26e8f-80a0-44d8-b46f-7407a3b11f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:18.339545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b26e8f-80a0-44d8-b46f-7407a3b11f47 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:18.339878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b26e8f-80a0-44d8-b46f-7407a3b11f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:18.340159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b26e8f-80a0-44d8-b46f-7407a3b11f47 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:18.342742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b26e8f-80a0-44d8-b46f-7407a3b11f47 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:18.344250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b26e8f-80a0-44d8-b46f-7407a3b11f47 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:18.347166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 93/171] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:18 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:18.348581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b26e8f-80a0-44d8-b46f-7407a3b11f47 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:18.443833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:18.448117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a638f0d-0bea-4d93-bb96-211f29595b54 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:18.490892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0934ce-5335-4146-b9ae-11664e90a5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:18.491555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0934ce-5335-4146-b9ae-11664e90a5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:18.491838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0934ce-5335-4146-b9ae-11664e90a5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:18.492117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0934ce-5335-4146-b9ae-11664e90a5e2 None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:18.493375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0934ce-5335-4146-b9ae-11664e90a5e2 None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:18.494964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0934ce-5335-4146-b9ae-11664e90a5e2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:18.496884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 79/172] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:18 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:18.497905 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0934ce-5335-4146-b9ae-11664e90a5e2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:18.572795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b26e8f-80a0-44d8-b46f-7407a3b11f47 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:18.647775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974831de-613a-4299-be26-2c513420a56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:18.648377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974831de-613a-4299-be26-2c513420a56d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:18.648724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974831de-613a-4299-be26-2c513420a56d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:18.648978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974831de-613a-4299-be26-2c513420a56d None admin] PATH_INFO: `/v3/users/49483f572c9c483b9311fe1b812f3c38` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:18.653601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974831de-613a-4299-be26-2c513420a56d None admin] RBAC: Authorizing `identity:get_user(user_id=49483f572c9c483b9311fe1b812f3c38)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:18.655074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974831de-613a-4299-be26-2c513420a56d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:18.656715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 94/173] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:18 2022] GET /identity/v3/users/49483f572c9c483b9311fe1b812f3c38 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:18.661281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0934ce-5335-4146-b9ae-11664e90a5e2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:18.662428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974831de-613a-4299-be26-2c513420a56d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:18.725318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed885-b71e-4f25-a0d2-ad9b8b7eac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:18.725318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed885-b71e-4f25-a0d2-ad9b8b7eac4b None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:18.725318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed885-b71e-4f25-a0d2-ad9b8b7eac4b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:18.725318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed885-b71e-4f25-a0d2-ad9b8b7eac4b None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/users/722cc12185804086810585b612f1e991/roles/3cc2fe72d0a245238f0d8df6763755c9` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:18.725742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ed885-b71e-4f25-a0d2-ad9b8b7eac4b None admin] RBAC: Authorizing `identity:create_grant(project_id=89d1fa86df9a40fe96f2a7554edbbaed, user_id=722cc12185804086810585b612f1e991, role_id=3cc2fe72d0a245238f0d8df6763755c9)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:18.725742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ed885-b71e-4f25-a0d2-ad9b8b7eac4b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:18.727304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:18.753506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 80/174] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:18 2022] PUT /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/users/722cc12185804086810585b612f1e991/roles/3cc2fe72d0a245238f0d8df6763755c9 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:18.754496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed885-b71e-4f25-a0d2-ad9b8b7eac4b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:18.808450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974831de-613a-4299-be26-2c513420a56d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:18.836860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:18.852603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678da1e-461a-4c56-a3d4-d87f8523baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:18.853311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678da1e-461a-4c56-a3d4-d87f8523baa6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:18.853595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678da1e-461a-4c56-a3d4-d87f8523baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:18.853869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678da1e-461a-4c56-a3d4-d87f8523baa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:19.003551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed885-b71e-4f25-a0d2-ad9b8b7eac4b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:19.005317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678da1e-461a-4c56-a3d4-d87f8523baa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:19.007086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678da1e-461a-4c56-a3d4-d87f8523baa6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:19.068638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:19.089573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdebced-fcbc-44f2-97c2-1c4f2ec4daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:19.090383 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdebced-fcbc-44f2-97c2-1c4f2ec4daea None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:19.090769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdebced-fcbc-44f2-97c2-1c4f2ec4daea None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:19.091146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdebced-fcbc-44f2-97c2-1c4f2ec4daea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:19.152831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 95/175] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:19.161054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678da1e-461a-4c56-a3d4-d87f8523baa6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:19.218513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdebced-fcbc-44f2-97c2-1c4f2ec4daea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:19.221425 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdebced-fcbc-44f2-97c2-1c4f2ec4daea None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:19.281988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:19.389140 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678da1e-461a-4c56-a3d4-d87f8523baa6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:19.433803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 81/176] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:19.438870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdebced-fcbc-44f2-97c2-1c4f2ec4daea None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:19.459304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:19.467592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefca6b-9078-414c-83d1-902254accd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:19.468258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefca6b-9078-414c-83d1-902254accd5c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:19.468596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefca6b-9078-414c-83d1-902254accd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:19.468875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefca6b-9078-414c-83d1-902254accd5c None admin] PATH_INFO: `/v3/projects/b0424e1775c04de0a035654307b41a02` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:19.470310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefca6b-9078-414c-83d1-902254accd5c None admin] RBAC: Authorizing `identity:get_project(project_id=b0424e1775c04de0a035654307b41a02)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:19.472306 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefca6b-9078-414c-83d1-902254accd5c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:19.479697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 96/177] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:19 2022] GET /identity/v3/projects/b0424e1775c04de0a035654307b41a02 => generated 312 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:19.491948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefca6b-9078-414c-83d1-902254accd5c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:19.677955 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefca6b-9078-414c-83d1-902254accd5c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:19.685145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdebced-fcbc-44f2-97c2-1c4f2ec4daea None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:19.730062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61212700-d86e-47f2-9aa4-2303337fb8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:19.730690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61212700-d86e-47f2-9aa4-2303337fb8cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:19.730972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61212700-d86e-47f2-9aa4-2303337fb8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:19.731244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61212700-d86e-47f2-9aa4-2303337fb8cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:19.732044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61212700-d86e-47f2-9aa4-2303337fb8cf None admin] RBAC: Adding query filter params (role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=b0424e1775c04de0a035654307b41a02, user.id=49483f572c9c483b9311fe1b812f3c38) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:19.732355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61212700-d86e-47f2-9aa4-2303337fb8cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:19.733833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61212700-d86e-47f2-9aa4-2303337fb8cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:19.744297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 97/178] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:19 2022] GET /identity/v3/role_assignments?user.id=49483f572c9c483b9311fe1b812f3c38&scope.project.id=b0424e1775c04de0a035654307b41a02&role.id=551979dba2834cf397f2c386c2c64b2d => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:19.745567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c019-9810-4416-9a39-92cf13b656ed None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:19.745854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c019-9810-4416-9a39-92cf13b656ed None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:19.746130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c019-9810-4416-9a39-92cf13b656ed None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:19.747360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 98/179] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:19.748388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:19.758777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce30d00-7eaa-45a7-91eb-9f0e932e212c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:19.759102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce30d00-7eaa-45a7-91eb-9f0e932e212c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:19.759332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce30d00-7eaa-45a7-91eb-9f0e932e212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:19.767068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ea004-12ba-45d0-869f-9ef076afc3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:19.767743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ea004-12ba-45d0-869f-9ef076afc3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:19.768027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ea004-12ba-45d0-869f-9ef076afc3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:19.768301 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ea004-12ba-45d0-869f-9ef076afc3a9 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:19.778375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ea004-12ba-45d0-869f-9ef076afc3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:19.779906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ea004-12ba-45d0-869f-9ef076afc3a9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:19.782092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 82/180] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:19 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:19.797750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5ce30d00-7eaa-45a7-91eb-9f0e932e212c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:19.798381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ea004-12ba-45d0-869f-9ef076afc3a9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:19.799049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5ce30d00-7eaa-45a7-91eb-9f0e932e212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:19.948781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:19.967647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 99/181] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:20.011248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce30d00-7eaa-45a7-91eb-9f0e932e212c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:20.025187 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ea004-12ba-45d0-869f-9ef076afc3a9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.064868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af6443-9dd8-4454-a707-521c62a34477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.065517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af6443-9dd8-4454-a707-521c62a34477 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.065796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af6443-9dd8-4454-a707-521c62a34477 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.066635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af6443-9dd8-4454-a707-521c62a34477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.067543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af6443-9dd8-4454-a707-521c62a34477 None admin] RBAC: Adding query filter params (role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed, user.id=49483f572c9c483b9311fe1b812f3c38) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:20.067852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af6443-9dd8-4454-a707-521c62a34477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:20.069321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af6443-9dd8-4454-a707-521c62a34477 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:20.076104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 83/182] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:19 2022] GET /identity/v3/role_assignments?user.id=49483f572c9c483b9311fe1b812f3c38&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=74f371799fd245bf987e798f5c342e6a => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:20.166475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab97e1f-ca3b-4398-8632-17ad28b118e4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.166791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab97e1f-ca3b-4398-8632-17ad28b118e4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.167023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab97e1f-ca3b-4398-8632-17ad28b118e4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.168374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 84/183] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:20.174624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f653e3-023f-4c9d-befa-c324417a0d28 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.174922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f653e3-023f-4c9d-befa-c324417a0d28 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.175183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f653e3-023f-4c9d-befa-c324417a0d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.199914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-22f653e3-023f-4c9d-befa-c324417a0d28 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:20.201047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-22f653e3-023f-4c9d-befa-c324417a0d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:20.217498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce30d00-7eaa-45a7-91eb-9f0e932e212c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.274337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df56a1-0a92-454b-b089-981e88533e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.275025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df56a1-0a92-454b-b089-981e88533e7f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.275315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df56a1-0a92-454b-b089-981e88533e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.275599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df56a1-0a92-454b-b089-981e88533e7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.296184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df56a1-0a92-454b-b089-981e88533e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f7df56a1-0a92-454b-b089-981e88533e7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:20.302387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:20.303303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:20.304096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 100/184] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:20.308837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:20.310676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df56a1-0a92-454b-b089-981e88533e7f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:20.329046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 85/185] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:20.330028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe062c-99dc-4b4c-b1c8-52ead6967b58 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.330721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe062c-99dc-4b4c-b1c8-52ead6967b58 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.331014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe062c-99dc-4b4c-b1c8-52ead6967b58 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.333259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 86/186] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:20.339265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397dcf1-2eaa-4093-8157-4d51cbffc056 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.339558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397dcf1-2eaa-4093-8157-4d51cbffc056 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.339842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397dcf1-2eaa-4093-8157-4d51cbffc056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.364308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2397dcf1-2eaa-4093-8157-4d51cbffc056 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:20.365776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2397dcf1-2eaa-4093-8157-4d51cbffc056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:20.394115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:20.443699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 87/187] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:20.444379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397dcf1-2eaa-4093-8157-4d51cbffc056 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:20.446881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df56a1-0a92-454b-b089-981e88533e7f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.485798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.486542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.486924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.487230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.487917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:20.488853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:20.491565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:20.505330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 101/188] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:20.505330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:20.568594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397dcf1-2eaa-4093-8157-4d51cbffc056 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.590645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:20.605834 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f7205-05ac-4251-8551-eb51685c098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.606475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f7205-05ac-4251-8551-eb51685c098c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.606755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f7205-05ac-4251-8551-eb51685c098c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.607026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f7205-05ac-4251-8551-eb51685c098c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.613351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a86e3-3594-4e05-94b7-a65546156e6b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.617636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f7205-05ac-4251-8551-eb51685c098c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-204f7205-05ac-4251-8551-eb51685c098c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:20.622758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:20.623809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 88/189] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:20.624591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f7205-05ac-4251-8551-eb51685c098c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:20.637399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:20.660992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98a5a5-2ac3-47b2-8fe3-0b6b2e96fa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.661919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98a5a5-2ac3-47b2-8fe3-0b6b2e96fa8c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.662198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98a5a5-2ac3-47b2-8fe3-0b6b2e96fa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.662466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98a5a5-2ac3-47b2-8fe3-0b6b2e96fa8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.663620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98a5a5-2ac3-47b2-8fe3-0b6b2e96fa8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:20.665075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98a5a5-2ac3-47b2-8fe3-0b6b2e96fa8c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:20.666823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 102/190] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:20.667406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98a5a5-2ac3-47b2-8fe3-0b6b2e96fa8c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:20.747156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f7205-05ac-4251-8551-eb51685c098c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.772921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98a5a5-2ac3-47b2-8fe3-0b6b2e96fa8c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.784621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e698d-3ce4-4836-a0e3-c83b75e0cef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.785696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e698d-3ce4-4836-a0e3-c83b75e0cef4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.786227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e698d-3ce4-4836-a0e3-c83b75e0cef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.786677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e698d-3ce4-4836-a0e3-c83b75e0cef4 None admin] PATH_INFO: `/v3/users/28251880c2674cdcb141ba28c12514a7` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.802832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e698d-3ce4-4836-a0e3-c83b75e0cef4 None admin] RBAC: Authorizing `identity:get_user(user_id=28251880c2674cdcb141ba28c12514a7)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:20.804459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e698d-3ce4-4836-a0e3-c83b75e0cef4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:20.806066 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.806498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 89/191] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:20 2022] GET /identity/v3/users/28251880c2674cdcb141ba28c12514a7 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:20.810101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.810101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.810101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.810101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:20.810101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:20.810101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:20.811875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e698d-3ce4-4836-a0e3-c83b75e0cef4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:20.818300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 103/192] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:20.819085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:20.855912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:20.869096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:20.922766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e698d-3ce4-4836-a0e3-c83b75e0cef4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.925194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7193ca-0f61-4144-928d-7571fafab6f3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:20.961128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7f4be-7a4d-4a36-8a38-649bc803162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.961891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7f4be-7a4d-4a36-8a38-649bc803162a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.962245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7f4be-7a4d-4a36-8a38-649bc803162a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.962531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7f4be-7a4d-4a36-8a38-649bc803162a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.963351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8444fab-ae2d-49b0-b366-d333058aa046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:20.963973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8444fab-ae2d-49b0-b366-d333058aa046 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:20.964258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8444fab-ae2d-49b0-b366-d333058aa046 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:20.964532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8444fab-ae2d-49b0-b366-d333058aa046 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:20.965230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8444fab-ae2d-49b0-b366-d333058aa046 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:20.966700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8444fab-ae2d-49b0-b366-d333058aa046 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:20.997425 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 90/193] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:20 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:20.997425 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8444fab-ae2d-49b0-b366-d333058aa046 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:21.053781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7f4be-7a4d-4a36-8a38-649bc803162a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:21.055394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7f4be-7a4d-4a36-8a38-649bc803162a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:21.110362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:21.141206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:21.150597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8444fab-ae2d-49b0-b366-d333058aa046 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:21.180757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 104/194] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:21.186878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7f4be-7a4d-4a36-8a38-649bc803162a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:21.198763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5aa3e-04ea-46a1-80c5-699f1605b6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:21.199663 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5aa3e-04ea-46a1-80c5-699f1605b6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:21.199942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5aa3e-04ea-46a1-80c5-699f1605b6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:21.200210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5aa3e-04ea-46a1-80c5-699f1605b6c2 None admin] PATH_INFO: `/v3/users/28251880c2674cdcb141ba28c12514a7` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:21.202156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5aa3e-04ea-46a1-80c5-699f1605b6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=28251880c2674cdcb141ba28c12514a7)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:21.203490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5aa3e-04ea-46a1-80c5-699f1605b6c2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:21.204619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 91/195] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:20 2022] GET /identity/v3/users/28251880c2674cdcb141ba28c12514a7 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:21.210387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5aa3e-04ea-46a1-80c5-699f1605b6c2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:21.323034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7f4be-7a4d-4a36-8a38-649bc803162a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:21.342106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5aa3e-04ea-46a1-80c5-699f1605b6c2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:21.373755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1f26c-8780-4be6-8289-6593a9b1e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:21.374694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1f26c-8780-4be6-8289-6593a9b1e142 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:21.375113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1f26c-8780-4be6-8289-6593a9b1e142 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:21.375527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1f26c-8780-4be6-8289-6593a9b1e142 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:21.376870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1f26c-8780-4be6-8289-6593a9b1e142 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:21.378510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1f26c-8780-4be6-8289-6593a9b1e142 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:21.380599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 105/196] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:21 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:21.386636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1f26c-8780-4be6-8289-6593a9b1e142 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:21.389126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e1865-435f-45fd-a82e-d5c7217e747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:21.389934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e1865-435f-45fd-a82e-d5c7217e747d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:21.390583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e1865-435f-45fd-a82e-d5c7217e747d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:21.390939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e1865-435f-45fd-a82e-d5c7217e747d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:21.413895 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:21.430260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:21.474461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e1865-435f-45fd-a82e-d5c7217e747d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:21.475963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e1865-435f-45fd-a82e-d5c7217e747d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:21.522116 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1f26c-8780-4be6-8289-6593a9b1e142 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:21.557984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5838263-65fe-462e-a900-dc38619a26b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:21.558609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838263-65fe-462e-a900-dc38619a26b4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:21.558888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838263-65fe-462e-a900-dc38619a26b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:21.559160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838263-65fe-462e-a900-dc38619a26b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:21.559953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5838263-65fe-462e-a900-dc38619a26b4 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1, user.id=28251880c2674cdcb141ba28c12514a7) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:21.560258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5838263-65fe-462e-a900-dc38619a26b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:21.561733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5838263-65fe-462e-a900-dc38619a26b4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:21.568098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 106/197] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:21 2022] GET /identity/v3/role_assignments?user.id=28251880c2674cdcb141ba28c12514a7&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:21.600669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd57f1-3e14-4bed-ad79-ba0c82f563a4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:21.600964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd57f1-3e14-4bed-ad79-ba0c82f563a4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:21.601184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd57f1-3e14-4bed-ad79-ba0c82f563a4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:21.601390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 92/198] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:21.602932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 107/199] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:21.608783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e1865-435f-45fd-a82e-d5c7217e747d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:21.613589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b6422-339c-4b8a-b27c-f69d1a8bd19c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:21.614054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b6422-339c-4b8a-b27c-f69d1a8bd19c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:21.614331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b6422-339c-4b8a-b27c-f69d1a8bd19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:21.651686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f25b6422-339c-4b8a-b27c-f69d1a8bd19c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:21.652891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f25b6422-339c-4b8a-b27c-f69d1a8bd19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:21.677855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:21.721230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e1865-435f-45fd-a82e-d5c7217e747d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:21.760595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc132a-348a-4137-8c04-493e36c62f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:21.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc132a-348a-4137-8c04-493e36c62f50 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:21.761580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc132a-348a-4137-8c04-493e36c62f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:21.761858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc132a-348a-4137-8c04-493e36c62f50 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:21.763142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc132a-348a-4137-8c04-493e36c62f50 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:21.764810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc132a-348a-4137-8c04-493e36c62f50 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:21.769720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 93/200] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:21 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:21.774274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc132a-348a-4137-8c04-493e36c62f50 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:21.794428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 108/201] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:21.830407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b6422-339c-4b8a-b27c-f69d1a8bd19c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:21.851795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:21.889693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc132a-348a-4137-8c04-493e36c62f50 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:21.926504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:21.927137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:21.927423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:21.928262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:21.933509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:21.934428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] RBAC: Adding query filter params (role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1, user.id=28251880c2674cdcb141ba28c12514a7) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:21.935042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:21.937798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:21.950464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 94/202] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:21 2022] GET /identity/v3/role_assignments?user.id=28251880c2674cdcb141ba28c12514a7&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=74f371799fd245bf987e798f5c342e6a => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:22.052597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b6422-339c-4b8a-b27c-f69d1a8bd19c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:22.104663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b0ecb-5b0d-40f8-b558-3d7ee5574235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:22.105501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0ecb-5b0d-40f8-b558-3d7ee5574235 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:22.105808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0ecb-5b0d-40f8-b558-3d7ee5574235 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:22.106083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0ecb-5b0d-40f8-b558-3d7ee5574235 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:22.121670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b0ecb-5b0d-40f8-b558-3d7ee5574235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-917b0ecb-5b0d-40f8-b558-3d7ee5574235 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:22.126742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:22.127746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:22.128947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 109/203] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:22.133857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:22.265327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d9ed5-7dfd-401d-9f88-571499816210 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:22.304745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:22.305488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:22.305816 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:22.306103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:22.306828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:22.307142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:22.308529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:22.317269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 95/204] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:22.322977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b0ecb-5b0d-40f8-b558-3d7ee5574235 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:22.414619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:22.465058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b0ecb-5b0d-40f8-b558-3d7ee5574235 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:22.515095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcef42-cd51-4f89-a0d8-08eec8a056bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:22.515750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcef42-cd51-4f89-a0d8-08eec8a056bb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:22.516036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcef42-cd51-4f89-a0d8-08eec8a056bb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:22.516314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcef42-cd51-4f89-a0d8-08eec8a056bb None admin] PATH_INFO: `/v3/users/722cc12185804086810585b612f1e991` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:22.522097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcef42-cd51-4f89-a0d8-08eec8a056bb None admin] RBAC: Authorizing `identity:get_user(user_id=722cc12185804086810585b612f1e991)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:22.523566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcef42-cd51-4f89-a0d8-08eec8a056bb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:22.525206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 110/205] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:22 2022] GET /identity/v3/users/722cc12185804086810585b612f1e991 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:22.530663 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:22.625882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:22.669949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c48bd5-0d85-4ef8-8afe-a6ab97d36c9b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:22.714891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1593b-5af3-416f-8141-64a3a96638c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:22.715872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1593b-5af3-416f-8141-64a3a96638c5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:22.716360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1593b-5af3-416f-8141-64a3a96638c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:22.716840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1593b-5af3-416f-8141-64a3a96638c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:22.799302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1593b-5af3-416f-8141-64a3a96638c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:22.800838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1593b-5af3-416f-8141-64a3a96638c5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:22.872465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:22.926088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 96/206] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:22.931523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcef42-cd51-4f89-a0d8-08eec8a056bb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:23.085844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcef42-cd51-4f89-a0d8-08eec8a056bb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:23.135839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38e683-5712-455f-8393-41e568fdc95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:23.136496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38e683-5712-455f-8393-41e568fdc95f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:23.136781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38e683-5712-455f-8393-41e568fdc95f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:23.137059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38e683-5712-455f-8393-41e568fdc95f None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:23.138273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38e683-5712-455f-8393-41e568fdc95f None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:23.139740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38e683-5712-455f-8393-41e568fdc95f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:23.141438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 111/207] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:22 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:23.147458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1593b-5af3-416f-8141-64a3a96638c5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:23.255440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1593b-5af3-416f-8141-64a3a96638c5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:23.290941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302120eb-e47d-4127-b7bd-09b8eb537606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:23.291665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302120eb-e47d-4127-b7bd-09b8eb537606 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:23.291847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302120eb-e47d-4127-b7bd-09b8eb537606 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:23.292126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302120eb-e47d-4127-b7bd-09b8eb537606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:23.292919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302120eb-e47d-4127-b7bd-09b8eb537606 None admin] RBAC: Adding query filter params (role.id=3cc2fe72d0a245238f0d8df6763755c9, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed, user.id=722cc12185804086810585b612f1e991) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:23.293228 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302120eb-e47d-4127-b7bd-09b8eb537606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:23.294696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302120eb-e47d-4127-b7bd-09b8eb537606 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:23.301647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 97/208] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:23 2022] GET /identity/v3/role_assignments?user.id=722cc12185804086810585b612f1e991&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=3cc2fe72d0a245238f0d8df6763755c9 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:23.640296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b049c-62bb-4e21-a94d-25a71e0fe79a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:23.640496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b049c-62bb-4e21-a94d-25a71e0fe79a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:23.640757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b049c-62bb-4e21-a94d-25a71e0fe79a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:23.642171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 112/209] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:23.648417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb437-666a-4f76-8708-91696c122b9f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:23.648714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb437-666a-4f76-8708-91696c122b9f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:23.648988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb437-666a-4f76-8708-91696c122b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:23.672325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2ffcb437-666a-4f76-8708-91696c122b9f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:23.673551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2ffcb437-666a-4f76-8708-91696c122b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:23.696325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:23.771284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 98/210] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:23.797244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b049c-62bb-4e21-a94d-25a71e0fe79a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:23.802202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:23.918004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b049c-62bb-4e21-a94d-25a71e0fe79a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:23.952982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec9108-6d37-4634-80ff-fb59a7e7a502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:23.953780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec9108-6d37-4634-80ff-fb59a7e7a502 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:23.953985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec9108-6d37-4634-80ff-fb59a7e7a502 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:23.954261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec9108-6d37-4634-80ff-fb59a7e7a502 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:23.955415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec9108-6d37-4634-80ff-fb59a7e7a502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:23.957532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec9108-6d37-4634-80ff-fb59a7e7a502 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:23.959137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 113/211] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:23.964802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb437-666a-4f76-8708-91696c122b9f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:23.985256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d4341-c4d1-4e4f-b627-930c77492152 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:23.985656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d4341-c4d1-4e4f-b627-930c77492152 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:23.985938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d4341-c4d1-4e4f-b627-930c77492152 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:23.987552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 114/212] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:23.992802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0388c2-44e3-43d8-86ea-0f1794ba748c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:23.993089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0388c2-44e3-43d8-86ea-0f1794ba748c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:23.993402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0388c2-44e3-43d8-86ea-0f1794ba748c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.011665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1f0388c2-44e3-43d8-86ea-0f1794ba748c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:24.012842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1f0388c2-44e3-43d8-86ea-0f1794ba748c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:24.074163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb437-666a-4f76-8708-91696c122b9f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:24.083251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 115/213] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:24.097791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb31ce-5025-4862-9036-b8da4a4ec39c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.097959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb31ce-5025-4862-9036-b8da4a4ec39c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.098241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb31ce-5025-4862-9036-b8da4a4ec39c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.099734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 116/214] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:24.105517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a31d70-4bfb-4224-b5e0-7991b3c3ff55 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.105870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a31d70-4bfb-4224-b5e0-7991b3c3ff55 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.106112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a31d70-4bfb-4224-b5e0-7991b3c3ff55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.112517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:24.113509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409e1fb-0ea7-4138-8a6e-fb930373ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:24.115195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409e1fb-0ea7-4138-8a6e-fb930373ee8f None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.115496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409e1fb-0ea7-4138-8a6e-fb930373ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.115976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409e1fb-0ea7-4138-8a6e-fb930373ee8f None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.116590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409e1fb-0ea7-4138-8a6e-fb930373ee8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:24.118083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409e1fb-0ea7-4138-8a6e-fb930373ee8f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:24.125428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e0a31d70-4bfb-4224-b5e0-7991b3c3ff55 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:24.126616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e0a31d70-4bfb-4224-b5e0-7991b3c3ff55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:24.142518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 99/215] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:23 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:24.143355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409e1fb-0ea7-4138-8a6e-fb930373ee8f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:24.157336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:24.205720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 117/216] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:24.229635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a31d70-4bfb-4224-b5e0-7991b3c3ff55 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:24.281663 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409e1fb-0ea7-4138-8a6e-fb930373ee8f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:24.322138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fea11-8ed4-4dd7-98e3-110fe5b5be6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:24.322879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fea11-8ed4-4dd7-98e3-110fe5b5be6b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.323061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fea11-8ed4-4dd7-98e3-110fe5b5be6b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.323334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fea11-8ed4-4dd7-98e3-110fe5b5be6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.337996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fea11-8ed4-4dd7-98e3-110fe5b5be6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ba9fea11-8ed4-4dd7-98e3-110fe5b5be6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:24.344347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:24.345279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:24.346143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 100/217] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:24.352779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fea11-8ed4-4dd7-98e3-110fe5b5be6b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:24.352944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a31d70-4bfb-4224-b5e0-7991b3c3ff55 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:24.397125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:24.403234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5420-e8a5-4150-8618-b7b2826df5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:24.405609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5420-e8a5-4150-8618-b7b2826df5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.406447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5420-e8a5-4150-8618-b7b2826df5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.406952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5420-e8a5-4150-8618-b7b2826df5f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.418182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef5420-e8a5-4150-8618-b7b2826df5f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a0ef5420-e8a5-4150-8618-b7b2826df5f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:24.424266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:24.425234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:24.426213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:24.428836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 118/218] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:24.434379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5420-e8a5-4150-8618-b7b2826df5f6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:24.438851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:24.486805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fea11-8ed4-4dd7-98e3-110fe5b5be6b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:24.536192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270de71-d6d0-403a-9bbe-d9fe5c36ba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:24.536822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270de71-d6d0-403a-9bbe-d9fe5c36ba00 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.537103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270de71-d6d0-403a-9bbe-d9fe5c36ba00 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.538341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270de71-d6d0-403a-9bbe-d9fe5c36ba00 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.538341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270de71-d6d0-403a-9bbe-d9fe5c36ba00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:24.538511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270de71-d6d0-403a-9bbe-d9fe5c36ba00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:24.539831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270de71-d6d0-403a-9bbe-d9fe5c36ba00 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:24.550890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 101/219] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:24.552195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18a10f-21db-4cbd-9c48-b398ead82906 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.552482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18a10f-21db-4cbd-9c48-b398ead82906 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.552753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18a10f-21db-4cbd-9c48-b398ead82906 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.554569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 102/220] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:24.557067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18a10f-21db-4cbd-9c48-b398ead82906 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:24.576967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5420-e8a5-4150-8618-b7b2826df5f6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:24.626017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472a7dd-a718-4b83-8214-fdee9467166a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:24.626647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472a7dd-a718-4b83-8214-fdee9467166a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.626979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472a7dd-a718-4b83-8214-fdee9467166a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.627258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472a7dd-a718-4b83-8214-fdee9467166a None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.627985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472a7dd-a718-4b83-8214-fdee9467166a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:24.628327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472a7dd-a718-4b83-8214-fdee9467166a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:24.631192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472a7dd-a718-4b83-8214-fdee9467166a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:24.642991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 119/221] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:24.648943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4721-0bc8-4f7c-8ba9-565e0737c9b8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.648943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4721-0bc8-4f7c-8ba9-565e0737c9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.648943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4721-0bc8-4f7c-8ba9-565e0737c9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.668138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c21a4721-0bc8-4f7c-8ba9-565e0737c9b8 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:24.669675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c21a4721-0bc8-4f7c-8ba9-565e0737c9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:24.695195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18a10f-21db-4cbd-9c48-b398ead82906 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:24.730801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:24.742211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5adc809-aad4-41b8-abfd-a4a728275575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:24.743117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:24.743880 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adc809-aad4-41b8-abfd-a4a728275575 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.744414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adc809-aad4-41b8-abfd-a4a728275575 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.744903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adc809-aad4-41b8-abfd-a4a728275575 None admin] PATH_INFO: `/v3/users/7cfe0d584975405c97afe5ba3302b7f0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.762041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5adc809-aad4-41b8-abfd-a4a728275575 None admin] RBAC: Authorizing `identity:get_user(user_id=7cfe0d584975405c97afe5ba3302b7f0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:24.763852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5adc809-aad4-41b8-abfd-a4a728275575 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:24.765138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 120/222] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:24.765830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a4721-0bc8-4f7c-8ba9-565e0737c9b8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:24.766617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 103/223] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:24 2022] GET /identity/v3/users/7cfe0d584975405c97afe5ba3302b7f0 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:24.772260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5adc809-aad4-41b8-abfd-a4a728275575 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:24.875125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a4721-0bc8-4f7c-8ba9-565e0737c9b8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:24.888130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5adc809-aad4-41b8-abfd-a4a728275575 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:24.919412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0929b-1f68-4388-9969-5493d99ae2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:24.919872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0929b-1f68-4388-9969-5493d99ae2de None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.920150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0929b-1f68-4388-9969-5493d99ae2de None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.920473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0929b-1f68-4388-9969-5493d99ae2de None admin] PATH_INFO: `/v3/users/28251880c2674cdcb141ba28c12514a7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.921720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0929b-1f68-4388-9969-5493d99ae2de None admin] RBAC: Authorizing `identity:get_user(user_id=28251880c2674cdcb141ba28c12514a7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:24.923141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0929b-1f68-4388-9969-5493d99ae2de None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:24.924835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 121/224] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:24 2022] GET /identity/v3/users/28251880c2674cdcb141ba28c12514a7 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:24.925767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0929b-1f68-4388-9969-5493d99ae2de None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:24.943446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d670-ac3a-4f89-91e5-b756822867e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:24.944730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d670-ac3a-4f89-91e5-b756822867e1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:24.945215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d670-ac3a-4f89-91e5-b756822867e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:24.945996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d670-ac3a-4f89-91e5-b756822867e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:24.990607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:25.015217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:25.032106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385d670-ac3a-4f89-91e5-b756822867e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:25.034142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385d670-ac3a-4f89-91e5-b756822867e1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:25.054857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0929b-1f68-4388-9969-5493d99ae2de None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:25.091286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc8aee-85e7-462f-a4e0-bfda333cdf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:25.092108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc8aee-85e7-462f-a4e0-bfda333cdf1d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:25.092492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc8aee-85e7-462f-a4e0-bfda333cdf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:25.092869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc8aee-85e7-462f-a4e0-bfda333cdf1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:25.105009 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc8aee-85e7-462f-a4e0-bfda333cdf1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-97cc8aee-85e7-462f-a4e0-bfda333cdf1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:25.111689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:25.112611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:25.114198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 122/225] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:25.114857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc8aee-85e7-462f-a4e0-bfda333cdf1d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:25.152744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 104/226] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:23:25.154000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d670-ac3a-4f89-91e5-b756822867e1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:25.225265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc8aee-85e7-462f-a4e0-bfda333cdf1d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:25.239390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:25.246312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:25.262202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d670-ac3a-4f89-91e5-b756822867e1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:25.263795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac9434-42b4-4310-b572-7e59efc9b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:25.264473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac9434-42b4-4310-b572-7e59efc9b886 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:25.264881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac9434-42b4-4310-b572-7e59efc9b886 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:25.265819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac9434-42b4-4310-b572-7e59efc9b886 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:25.266953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac9434-42b4-4310-b572-7e59efc9b886 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:25.268488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac9434-42b4-4310-b572-7e59efc9b886 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:25.270865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 123/227] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:25 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:25.271620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac9434-42b4-4310-b572-7e59efc9b886 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:25.302729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:25.303769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:25.304376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:25.304846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:25.305800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:25.306303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:25.307989 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:25.317968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 105/228] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:25.319245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:25.375872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac9434-42b4-4310-b572-7e59efc9b886 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:25.411568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96790e1d-09a8-43b6-a953-3553aaabbf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:25.412580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96790e1d-09a8-43b6-a953-3553aaabbf91 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:25.412580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96790e1d-09a8-43b6-a953-3553aaabbf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:25.412778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96790e1d-09a8-43b6-a953-3553aaabbf91 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:25.413927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96790e1d-09a8-43b6-a953-3553aaabbf91 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:25.415639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96790e1d-09a8-43b6-a953-3553aaabbf91 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:25.417492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 124/229] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:25 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:25.418154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96790e1d-09a8-43b6-a953-3553aaabbf91 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:25.445002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c312a3-4f17-4b7b-ad9f-4523f5599319 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:25.471052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:25.493331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084cba8-d193-421a-990e-848ea3bd02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:25.494462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084cba8-d193-421a-990e-848ea3bd02d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:25.494815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084cba8-d193-421a-990e-848ea3bd02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:25.495163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084cba8-d193-421a-990e-848ea3bd02d2 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/users/28251880c2674cdcb141ba28c12514a7/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:25.497232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084cba8-d193-421a-990e-848ea3bd02d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0fe9b2593d1947c19cb976bc9c6e51f1, user_id=28251880c2674cdcb141ba28c12514a7, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:25.498889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084cba8-d193-421a-990e-848ea3bd02d2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:25.516951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 106/230] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:25 2022] PUT /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/users/28251880c2674cdcb141ba28c12514a7/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:25.521516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084cba8-d193-421a-990e-848ea3bd02d2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:25.543455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:25.560459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96790e1d-09a8-43b6-a953-3553aaabbf91 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:25.636317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672278-98f2-4834-8fa3-c05db5d507e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:25.636984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672278-98f2-4834-8fa3-c05db5d507e0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:25.637264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672278-98f2-4834-8fa3-c05db5d507e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:25.637559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0672278-98f2-4834-8fa3-c05db5d507e0 None admin] PATH_INFO: `/v3/users/28251880c2674cdcb141ba28c12514a7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:25.640088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0672278-98f2-4834-8fa3-c05db5d507e0 None admin] RBAC: Authorizing `identity:get_user(user_id=28251880c2674cdcb141ba28c12514a7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:25.641687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0672278-98f2-4834-8fa3-c05db5d507e0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:25.643727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 125/231] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:25 2022] GET /identity/v3/users/28251880c2674cdcb141ba28c12514a7 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:25.649778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672278-98f2-4834-8fa3-c05db5d507e0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:25.739670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084cba8-d193-421a-990e-848ea3bd02d2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:25.776701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:25.777251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:25.777581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:25.777858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:25.778655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] RBAC: Adding query filter params (role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1, user.id=7cfe0d584975405c97afe5ba3302b7f0) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:25.778971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:25.780413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:25.786575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 107/232] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:25 2022] GET /identity/v3/role_assignments?user.id=7cfe0d584975405c97afe5ba3302b7f0&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=551979dba2834cf397f2c386c2c64b2d => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:25.811585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0672278-98f2-4834-8fa3-c05db5d507e0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:25.832435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:25.854129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453fc6-f08b-4ff8-ac18-d5cc5af09e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:25.855106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453fc6-f08b-4ff8-ac18-d5cc5af09e61 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:25.855536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453fc6-f08b-4ff8-ac18-d5cc5af09e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:25.855972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453fc6-f08b-4ff8-ac18-d5cc5af09e61 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:25.857382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453fc6-f08b-4ff8-ac18-d5cc5af09e61 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:25.859161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453fc6-f08b-4ff8-ac18-d5cc5af09e61 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:25.861057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 126/233] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:25 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:25.867326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:25.951839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:25.984759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ad23-c61e-4baf-bdc0-af323268a0a6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:26.020817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec89d8-7575-4c76-bec7-1d1a1f6a84e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:26.021526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec89d8-7575-4c76-bec7-1d1a1f6a84e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:26.021827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec89d8-7575-4c76-bec7-1d1a1f6a84e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:26.022129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec89d8-7575-4c76-bec7-1d1a1f6a84e7 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/users/28251880c2674cdcb141ba28c12514a7/roles/74f371799fd245bf987e798f5c342e6a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:26.024355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec89d8-7575-4c76-bec7-1d1a1f6a84e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0fe9b2593d1947c19cb976bc9c6e51f1, user_id=28251880c2674cdcb141ba28c12514a7, role_id=74f371799fd245bf987e798f5c342e6a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:26.025858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec89d8-7575-4c76-bec7-1d1a1f6a84e7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:26.043523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 108/234] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:25 2022] PUT /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/users/28251880c2674cdcb141ba28c12514a7/roles/74f371799fd245bf987e798f5c342e6a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:26.541710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ccd0-b841-46ff-9988-04849d838447 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:26.543382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ccd0-b841-46ff-9988-04849d838447 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:26.543665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ccd0-b841-46ff-9988-04849d838447 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:26.545325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 127/235] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:26.550947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a531b-3079-41f2-94e6-a660ea22601c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:26.552001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a531b-3079-41f2-94e6-a660ea22601c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:26.552001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a531b-3079-41f2-94e6-a660ea22601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:26.577677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b56a531b-3079-41f2-94e6-a660ea22601c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:26.578882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b56a531b-3079-41f2-94e6-a660ea22601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:26.671502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 109/236] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:26.682303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b7c6a-f647-4321-ac64-d02224dcfc7f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:26.682584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b7c6a-f647-4321-ac64-d02224dcfc7f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:26.682865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b7c6a-f647-4321-ac64-d02224dcfc7f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:26.684442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 128/237] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:26.689863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0515a-34fc-441e-8fdf-0a1268441a7b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:26.690158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0515a-34fc-441e-8fdf-0a1268441a7b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:26.690441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0515a-34fc-441e-8fdf-0a1268441a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:26.709841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e4e0515a-34fc-441e-8fdf-0a1268441a7b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:26.711101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e4e0515a-34fc-441e-8fdf-0a1268441a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:26.712242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b7c6a-f647-4321-ac64-d02224dcfc7f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:26.810141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:26.830749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 110/238] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:26.860186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0515a-34fc-441e-8fdf-0a1268441a7b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:26.935216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b7c6a-f647-4321-ac64-d02224dcfc7f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:26.983402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0515a-34fc-441e-8fdf-0a1268441a7b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:27.024164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7774bb1-4a3d-4d3d-8a4c-0475a68af6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:27.024823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7774bb1-4a3d-4d3d-8a4c-0475a68af6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:27.025107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7774bb1-4a3d-4d3d-8a4c-0475a68af6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:27.025408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7774bb1-4a3d-4d3d-8a4c-0475a68af6d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:27.037631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92dbad-c5e4-4c7b-a6fd-dd46fd872269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:27.038279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92dbad-c5e4-4c7b-a6fd-dd46fd872269 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:27.038565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92dbad-c5e4-4c7b-a6fd-dd46fd872269 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:27.038838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92dbad-c5e4-4c7b-a6fd-dd46fd872269 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:27.041737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92dbad-c5e4-4c7b-a6fd-dd46fd872269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:27.046528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92dbad-c5e4-4c7b-a6fd-dd46fd872269 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:27.048177 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 111/239] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:27.049115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7774bb1-4a3d-4d3d-8a4c-0475a68af6d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:27.054397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92dbad-c5e4-4c7b-a6fd-dd46fd872269 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e7774bb1-4a3d-4d3d-8a4c-0475a68af6d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:27.060595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:27.061783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:27.063542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 129/240] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:27.069802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7774bb1-4a3d-4d3d-8a4c-0475a68af6d6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:27.146355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:27.169672 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:27.213427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7774bb1-4a3d-4d3d-8a4c-0475a68af6d6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:27.255096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:27.256471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:27.256978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:27.257482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:27.258427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:27.258934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:27.260506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:27.271155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 130/241] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:27.272643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92dbad-c5e4-4c7b-a6fd-dd46fd872269 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:27.281766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:27.318595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66708b-d026-4b52-9f24-13daf7117afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:27.319234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66708b-d026-4b52-9f24-13daf7117afc None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:27.319521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66708b-d026-4b52-9f24-13daf7117afc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:27.319800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66708b-d026-4b52-9f24-13daf7117afc None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:27.321011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66708b-d026-4b52-9f24-13daf7117afc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:27.321985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66708b-d026-4b52-9f24-13daf7117afc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:27.392185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 112/242] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:27 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:23:27.403507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2aa0b2-c9da-4825-bbd2-240f7a864db1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:27.438209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:27.444295 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e662b65-2843-4594-a410-672d8dd608ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:27.445344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e662b65-2843-4594-a410-672d8dd608ea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:27.445897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e662b65-2843-4594-a410-672d8dd608ea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:27.446444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e662b65-2843-4594-a410-672d8dd608ea None admin] PATH_INFO: `/v3/users/eca95181948046ac843318606f071c37` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:27.463819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e662b65-2843-4594-a410-672d8dd608ea None admin] RBAC: Authorizing `identity:get_user(user_id=eca95181948046ac843318606f071c37)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:27.465755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e662b65-2843-4594-a410-672d8dd608ea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:27.468287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 131/243] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:27 2022] GET /identity/v3/users/eca95181948046ac843318606f071c37 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:27.478494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66708b-d026-4b52-9f24-13daf7117afc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:27.616329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66708b-d026-4b52-9f24-13daf7117afc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:27.635225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:27.660594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500512a4-31f2-49f9-a36a-bc04b1bd2169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:27.661330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500512a4-31f2-49f9-a36a-bc04b1bd2169 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:27.661619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500512a4-31f2-49f9-a36a-bc04b1bd2169 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:27.661904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500512a4-31f2-49f9-a36a-bc04b1bd2169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:27.753511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500512a4-31f2-49f9-a36a-bc04b1bd2169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:27.755262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500512a4-31f2-49f9-a36a-bc04b1bd2169 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:27.879877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 113/244] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:27.885559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e662b65-2843-4594-a410-672d8dd608ea None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:27.997175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e662b65-2843-4594-a410-672d8dd608ea None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:28.018756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4c0e3-3c06-4948-9edb-144a5055e07f None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.019502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4c0e3-3c06-4948-9edb-144a5055e07f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.020009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4c0e3-3c06-4948-9edb-144a5055e07f None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.025609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 114/245] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:28.027403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634732-577a-47ac-8a35-08d2111ace06 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.027896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634732-577a-47ac-8a35-08d2111ace06 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.028373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634732-577a-47ac-8a35-08d2111ace06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.045678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1544c-11e6-4959-9c29-9af48703bd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:28.046686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1544c-11e6-4959-9c29-9af48703bd28 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.047196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1544c-11e6-4959-9c29-9af48703bd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.047673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1544c-11e6-4959-9c29-9af48703bd28 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.049131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1544c-11e6-4959-9c29-9af48703bd28 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:28.051085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1544c-11e6-4959-9c29-9af48703bd28 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:28.053082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 132/246] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:27 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:28.054085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4a634732-577a-47ac-8a35-08d2111ace06 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:28.055390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4a634732-577a-47ac-8a35-08d2111ace06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:28.059116 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1544c-11e6-4959-9c29-9af48703bd28 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:28.085718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:28.104101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:28.144644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 115/247] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:28.171925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a634732-577a-47ac-8a35-08d2111ace06 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:28.184796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1544c-11e6-4959-9c29-9af48703bd28 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:28.219870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:28.220507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.221383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.221383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.221872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] RBAC: Adding query filter params (role.id=3cc2fe72d0a245238f0d8df6763755c9, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1, user.id=eca95181948046ac843318606f071c37) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:28.222711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:28.224183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:28.230056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 133/248] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:28 2022] GET /identity/v3/role_assignments?user.id=eca95181948046ac843318606f071c37&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=3cc2fe72d0a245238f0d8df6763755c9 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:28.290527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a634732-577a-47ac-8a35-08d2111ace06 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:28.330589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8e6e0-74a8-4027-af1a-3f3a1d1f4659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:28.332395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8e6e0-74a8-4027-af1a-3f3a1d1f4659 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.332884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8e6e0-74a8-4027-af1a-3f3a1d1f4659 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.333419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8e6e0-74a8-4027-af1a-3f3a1d1f4659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.345903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8e6e0-74a8-4027-af1a-3f3a1d1f4659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c3b8e6e0-74a8-4027-af1a-3f3a1d1f4659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:28.351774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:28.352749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:28.353593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:28.356113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 116/249] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:28.366453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:28.494207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:28.500026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861eb7-de29-41e8-8deb-9428956b0fa2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:28.537381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd47763-a15c-4b9f-9ba7-b1f4b09539c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:28.538013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd47763-a15c-4b9f-9ba7-b1f4b09539c5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.538300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd47763-a15c-4b9f-9ba7-b1f4b09539c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.538575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd47763-a15c-4b9f-9ba7-b1f4b09539c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.539590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd47763-a15c-4b9f-9ba7-b1f4b09539c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:28.539903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd47763-a15c-4b9f-9ba7-b1f4b09539c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:28.541287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd47763-a15c-4b9f-9ba7-b1f4b09539c5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:28.552492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 134/250] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:28.558284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8e6e0-74a8-4027-af1a-3f3a1d1f4659 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:28.566410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edae08-bf04-4930-bf0c-86f1e54e7170 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.566410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edae08-bf04-4930-bf0c-86f1e54e7170 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.566410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edae08-bf04-4930-bf0c-86f1e54e7170 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.567725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 135/251] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:28.573645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:28.577835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576694eb-6268-417c-99e8-99a23d4ec38d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.578130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576694eb-6268-417c-99e8-99a23d4ec38d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.578412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576694eb-6268-417c-99e8-99a23d4ec38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.597457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-576694eb-6268-417c-99e8-99a23d4ec38d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:28.598661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-576694eb-6268-417c-99e8-99a23d4ec38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:28.670689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 136/252] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:28.719213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576694eb-6268-417c-99e8-99a23d4ec38d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:28.729976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8e6e0-74a8-4027-af1a-3f3a1d1f4659 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:28.773332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abdfe0-cf93-4dc5-99da-50e2f918a8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:28.774247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abdfe0-cf93-4dc5-99da-50e2f918a8cb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.774533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abdfe0-cf93-4dc5-99da-50e2f918a8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.774811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abdfe0-cf93-4dc5-99da-50e2f918a8cb None admin] PATH_INFO: `/v3/users/28251880c2674cdcb141ba28c12514a7` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.776028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abdfe0-cf93-4dc5-99da-50e2f918a8cb None admin] RBAC: Authorizing `identity:get_user(user_id=28251880c2674cdcb141ba28c12514a7)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:28.778316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abdfe0-cf93-4dc5-99da-50e2f918a8cb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:28.780434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 117/253] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:28 2022] GET /identity/v3/users/28251880c2674cdcb141ba28c12514a7 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:28.786947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abdfe0-cf93-4dc5-99da-50e2f918a8cb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:28.875904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:28.895108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:28.916114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576694eb-6268-417c-99e8-99a23d4ec38d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:28.923085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abdfe0-cf93-4dc5-99da-50e2f918a8cb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:28.987509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979c1b2-6aad-4ec4-907e-23293ed77722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:28.988850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c1b2-6aad-4ec4-907e-23293ed77722 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.989136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c1b2-6aad-4ec4-907e-23293ed77722 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.989438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c1b2-6aad-4ec4-907e-23293ed77722 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:28.992575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb233675-9a26-43be-be06-73fa56c9fd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:28.993271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb233675-9a26-43be-be06-73fa56c9fd69 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:28.993575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb233675-9a26-43be-be06-73fa56c9fd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:28.993896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb233675-9a26-43be-be06-73fa56c9fd69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.013962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979c1b2-6aad-4ec4-907e-23293ed77722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2979c1b2-6aad-4ec4-907e-23293ed77722 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:29.021155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:29.023267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 137/254] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:29.025197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595c52a-0bd1-4e7d-9317-8102823a3633 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:29.026930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595c52a-0bd1-4e7d-9317-8102823a3633 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:29.026930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595c52a-0bd1-4e7d-9317-8102823a3633 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.026930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 138/255] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:29.030636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595c52a-0bd1-4e7d-9317-8102823a3633 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:29.133385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb233675-9a26-43be-be06-73fa56c9fd69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:29.134875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb233675-9a26-43be-be06-73fa56c9fd69 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:29.218184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595c52a-0bd1-4e7d-9317-8102823a3633 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:29.263216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d51dd-3e82-4d69-aff6-54e36b899ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:29.263763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d51dd-3e82-4d69-aff6-54e36b899ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:29.264051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d51dd-3e82-4d69-aff6-54e36b899ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:29.264331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d51dd-3e82-4d69-aff6-54e36b899ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.265165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d51dd-3e82-4d69-aff6-54e36b899ee8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:29.265504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d51dd-3e82-4d69-aff6-54e36b899ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:29.266911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d51dd-3e82-4d69-aff6-54e36b899ee8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:29.275821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 139/256] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:29.277173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae7928-d090-4c3f-8f57-f8176c00f1a2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:29.277500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae7928-d090-4c3f-8f57-f8176c00f1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:29.277781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae7928-d090-4c3f-8f57-f8176c00f1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.297319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:29.297841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9aae7928-d090-4c3f-8f57-f8176c00f1a2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:29.299125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9aae7928-d090-4c3f-8f57-f8176c00f1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:29.303609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:29.347578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 118/257] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:23:29.348460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb233675-9a26-43be-be06-73fa56c9fd69 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:29.413388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 140/258] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:29.414282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae7928-d090-4c3f-8f57-f8176c00f1a2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:29.528021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae7928-d090-4c3f-8f57-f8176c00f1a2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:29.553933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb233675-9a26-43be-be06-73fa56c9fd69 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:29.565385 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6329e-3245-4f0e-bb19-437c9fb36c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:29.566477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6329e-3245-4f0e-bb19-437c9fb36c76 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:29.566762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6329e-3245-4f0e-bb19-437c9fb36c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:29.567135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6329e-3245-4f0e-bb19-437c9fb36c76 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.568326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6329e-3245-4f0e-bb19-437c9fb36c76 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:29.569811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6329e-3245-4f0e-bb19-437c9fb36c76 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:29.571486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 141/259] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:29 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:29.572248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6329e-3245-4f0e-bb19-437c9fb36c76 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:29.592462 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:29.626600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaeda47-d671-4565-b819-5935d373f047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:29.627271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaeda47-d671-4565-b819-5935d373f047 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:29.627556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaeda47-d671-4565-b819-5935d373f047 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:29.627829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaeda47-d671-4565-b819-5935d373f047 None admin] PATH_INFO: `/v3/users/7cfe0d584975405c97afe5ba3302b7f0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.634027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaeda47-d671-4565-b819-5935d373f047 None admin] RBAC: Authorizing `identity:get_user(user_id=7cfe0d584975405c97afe5ba3302b7f0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:29.635496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaeda47-d671-4565-b819-5935d373f047 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:29.637128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 119/260] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:29 2022] GET /identity/v3/users/7cfe0d584975405c97afe5ba3302b7f0 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:29.637919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaeda47-d671-4565-b819-5935d373f047 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:29.683331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6329e-3245-4f0e-bb19-437c9fb36c76 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:29.715997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:29.721223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91644f7a-8f2a-4d4a-9b46-cd978fe83f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:29.721897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91644f7a-8f2a-4d4a-9b46-cd978fe83f83 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:29.722741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91644f7a-8f2a-4d4a-9b46-cd978fe83f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:29.723046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91644f7a-8f2a-4d4a-9b46-cd978fe83f83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.734953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91644f7a-8f2a-4d4a-9b46-cd978fe83f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-91644f7a-8f2a-4d4a-9b46-cd978fe83f83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:29.740485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:29.742560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 142/261] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:29.744570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91644f7a-8f2a-4d4a-9b46-cd978fe83f83 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:29.818212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:29.826020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaeda47-d671-4565-b819-5935d373f047 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:29.851514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91644f7a-8f2a-4d4a-9b46-cd978fe83f83 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:29.887558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe13fa-4d83-4c6b-a4b1-91a32f0cdef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:29.888209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe13fa-4d83-4c6b-a4b1-91a32f0cdef5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:29.888496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe13fa-4d83-4c6b-a4b1-91a32f0cdef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:29.888776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe13fa-4d83-4c6b-a4b1-91a32f0cdef5 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.889982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe13fa-4d83-4c6b-a4b1-91a32f0cdef5 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:29.891449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe13fa-4d83-4c6b-a4b1-91a32f0cdef5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:29.893245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 143/262] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:29 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:29.894867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe13fa-4d83-4c6b-a4b1-91a32f0cdef5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:29.934021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:29.935042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:29.935520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:29.935988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:29.936976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1, user.id=28251880c2674cdcb141ba28c12514a7) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:29.937521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:29.939171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:29.955787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 120/263] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:29 2022] GET /identity/v3/role_assignments?user.id=28251880c2674cdcb141ba28c12514a7&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:29.957557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:30.026950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe13fa-4d83-4c6b-a4b1-91a32f0cdef5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:30.073411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:30.073411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.073411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.073411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.073800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:30.073800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:30.073800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:30.079039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 144/264] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:30.079399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:30.086404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:30.199394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bb8cf-c0fd-442b-aff9-8d4112e7be42 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:30.210442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:30.277493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907af7e-4707-43ca-9107-04dc8093b9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:30.278671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907af7e-4707-43ca-9107-04dc8093b9ae None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.280098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907af7e-4707-43ca-9107-04dc8093b9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.280739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907af7e-4707-43ca-9107-04dc8093b9ae None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/users/7cfe0d584975405c97afe5ba3302b7f0/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.285898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907af7e-4707-43ca-9107-04dc8093b9ae None admin] RBAC: Authorizing `identity:create_grant(project_id=0fe9b2593d1947c19cb976bc9c6e51f1, user_id=7cfe0d584975405c97afe5ba3302b7f0, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:30.288949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907af7e-4707-43ca-9107-04dc8093b9ae None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:30.302502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ea0e1-990e-49cf-b75b-971a7adfff01 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:30.328634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 121/265] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:29 2022] PUT /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/users/7cfe0d584975405c97afe5ba3302b7f0/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:30.349068 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69586b-efde-4125-8543-c0baa5445ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:30.349778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69586b-efde-4125-8543-c0baa5445ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.350060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69586b-efde-4125-8543-c0baa5445ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.350335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69586b-efde-4125-8543-c0baa5445ff8 None admin] PATH_INFO: `/v3/users/28251880c2674cdcb141ba28c12514a7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.352950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69586b-efde-4125-8543-c0baa5445ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=28251880c2674cdcb141ba28c12514a7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:30.354490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69586b-efde-4125-8543-c0baa5445ff8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:30.356318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 145/266] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:30 2022] GET /identity/v3/users/28251880c2674cdcb141ba28c12514a7 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:30.362065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907af7e-4707-43ca-9107-04dc8093b9ae None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:30.499709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907af7e-4707-43ca-9107-04dc8093b9ae None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:30.541148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488de54-04a4-4406-8f1c-476381dbbcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:30.541838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488de54-04a4-4406-8f1c-476381dbbcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.542126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488de54-04a4-4406-8f1c-476381dbbcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.542400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488de54-04a4-4406-8f1c-476381dbbcf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.588874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:30.616334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58e298-a46b-490d-9661-babf64bd2b3c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.616619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58e298-a46b-490d-9661-babf64bd2b3c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.616889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58e298-a46b-490d-9661-babf64bd2b3c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.618522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 146/267] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:30.624415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f9e67-5df1-4cee-ab6a-c690fe9a554f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.624698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f9e67-5df1-4cee-ab6a-c690fe9a554f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.625060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f9e67-5df1-4cee-ab6a-c690fe9a554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.635746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488de54-04a4-4406-8f1c-476381dbbcf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:30.637580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488de54-04a4-4406-8f1c-476381dbbcf6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:30.650640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-096f9e67-5df1-4cee-ab6a-c690fe9a554f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:30.651795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-096f9e67-5df1-4cee-ab6a-c690fe9a554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:30.751806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:30.758690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 122/268] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:30.760188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d531f-c1f5-468e-9a1f-429f28b87388 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.760585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d531f-c1f5-468e-9a1f-429f28b87388 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.760858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d531f-c1f5-468e-9a1f-429f28b87388 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.761685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 147/269] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:30.762250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 123/270] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:30.766719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f9e67-5df1-4cee-ab6a-c690fe9a554f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:30.767577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbea16b-e573-4241-86a7-27a7e3b7ba41 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.767868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbea16b-e573-4241-86a7-27a7e3b7ba41 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.768147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbea16b-e573-4241-86a7-27a7e3b7ba41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.786008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-adbea16b-e573-4241-86a7-27a7e3b7ba41 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:30.787375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-adbea16b-e573-4241-86a7-27a7e3b7ba41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:30.864593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:30.872704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 124/271] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:30.873859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbea16b-e573-4241-86a7-27a7e3b7ba41 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:30.886155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f9e67-5df1-4cee-ab6a-c690fe9a554f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:30.942437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b81fc-b61c-4a38-9a70-e38e7375fa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:30.943326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b81fc-b61c-4a38-9a70-e38e7375fa63 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:30.943806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b81fc-b61c-4a38-9a70-e38e7375fa63 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:30.944229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b81fc-b61c-4a38-9a70-e38e7375fa63 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:30.945618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b81fc-b61c-4a38-9a70-e38e7375fa63 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:30.947270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b81fc-b61c-4a38-9a70-e38e7375fa63 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:30.949472 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 148/272] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:30 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:30.950384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b81fc-b61c-4a38-9a70-e38e7375fa63 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:30.982645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbea16b-e573-4241-86a7-27a7e3b7ba41 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.016835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:31.022454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a975a-1e6e-428c-94d6-5b0c1ece1d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.023105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a975a-1e6e-428c-94d6-5b0c1ece1d21 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.023652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a975a-1e6e-428c-94d6-5b0c1ece1d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.023982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a975a-1e6e-428c-94d6-5b0c1ece1d21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.036275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a975a-1e6e-428c-94d6-5b0c1ece1d21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e42a975a-1e6e-428c-94d6-5b0c1ece1d21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:31.041495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:31.042569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 125/273] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:31.043508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a975a-1e6e-428c-94d6-5b0c1ece1d21 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:31.065516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b81fc-b61c-4a38-9a70-e38e7375fa63 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.102794 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab028a6-6ef0-4cb2-947f-c3732402b85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.103713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab028a6-6ef0-4cb2-947f-c3732402b85b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.104152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab028a6-6ef0-4cb2-947f-c3732402b85b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.104571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab028a6-6ef0-4cb2-947f-c3732402b85b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.105886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab028a6-6ef0-4cb2-947f-c3732402b85b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:31.107481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab028a6-6ef0-4cb2-947f-c3732402b85b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:31.109215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 149/274] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:31.113780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:31.115046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab028a6-6ef0-4cb2-947f-c3732402b85b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:31.161327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a975a-1e6e-428c-94d6-5b0c1ece1d21 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.200443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.201067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.201422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.201916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.202487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] RBAC: Adding query filter params (role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1, user.id=28251880c2674cdcb141ba28c12514a7) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:31.202797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:31.204233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:31.210754 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 126/275] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:31 2022] GET /identity/v3/role_assignments?user.id=28251880c2674cdcb141ba28c12514a7&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=74f371799fd245bf987e798f5c342e6a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:31.211403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:31.228774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab028a6-6ef0-4cb2-947f-c3732402b85b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.273939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:31.280579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f13701-8259-4b9b-9a28-544c63a0c24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.282300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f13701-8259-4b9b-9a28-544c63a0c24a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.282928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f13701-8259-4b9b-9a28-544c63a0c24a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.283298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f13701-8259-4b9b-9a28-544c63a0c24a None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.284539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f13701-8259-4b9b-9a28-544c63a0c24a None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:31.285637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f13701-8259-4b9b-9a28-544c63a0c24a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:31.288161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f13701-8259-4b9b-9a28-544c63a0c24a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:31.301786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 150/276] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:31.303729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f5df-88da-42d4-bd9e-57f7edd050e5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.304126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f5df-88da-42d4-bd9e-57f7edd050e5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.304523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838f5df-88da-42d4-bd9e-57f7edd050e5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.311793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 151/277] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:31.313601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838f5df-88da-42d4-bd9e-57f7edd050e5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:31.347937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1311f9-90dd-4963-80ab-c180689b2da5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.376470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:31.396493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf17d3-2c60-41de-93e1-94c079e5d525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.397122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf17d3-2c60-41de-93e1-94c079e5d525 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.397436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf17d3-2c60-41de-93e1-94c079e5d525 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.397817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf17d3-2c60-41de-93e1-94c079e5d525 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.399641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf17d3-2c60-41de-93e1-94c079e5d525 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:31.400276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf17d3-2c60-41de-93e1-94c079e5d525 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:31.427166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 127/278] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:31 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:31.428980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97567-9825-49be-ab2b-3d855fc81436 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.429268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97567-9825-49be-ab2b-3d855fc81436 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.429578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97567-9825-49be-ab2b-3d855fc81436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.454410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838f5df-88da-42d4-bd9e-57f7edd050e5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.460819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c1b97567-9825-49be-ab2b-3d855fc81436 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:31.462138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c1b97567-9825-49be-ab2b-3d855fc81436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:31.504305 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f16b3-a934-4589-aa26-11be39c43ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.505102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f16b3-a934-4589-aa26-11be39c43ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.505410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f16b3-a934-4589-aa26-11be39c43ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.505724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f16b3-a934-4589-aa26-11be39c43ff9 None admin] PATH_INFO: `/v3/users/0abb4fab27d04964a9811abe54f87081` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.524192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f16b3-a934-4589-aa26-11be39c43ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=0abb4fab27d04964a9811abe54f87081)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:31.525880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f16b3-a934-4589-aa26-11be39c43ff9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:31.528122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 152/279] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:31 2022] GET /identity/v3/users/0abb4fab27d04964a9811abe54f87081 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:31.534563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f16b3-a934-4589-aa26-11be39c43ff9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:31.551185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 128/280] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:31.574095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b97567-9825-49be-ab2b-3d855fc81436 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:31.599074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:31.652894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f16b3-a934-4589-aa26-11be39c43ff9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.695272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b97567-9825-49be-ab2b-3d855fc81436 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.703454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.704195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:31.704773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.705152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.705479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.706206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] RBAC: Adding query filter params (role.id=551979dba2834cf397f2c386c2c64b2d, user.id=0abb4fab27d04964a9811abe54f87081, scope.system=all) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:31.706654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:31.708174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:31.715574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 153/281] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Jul 28 06:23:31 2022] GET /identity/v3/role_assignments?user.id=0abb4fab27d04964a9811abe54f87081&scope.system=all&role.id=551979dba2834cf397f2c386c2c64b2d => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:31.745808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1a6af-19bb-4666-b9dd-e6a8fff9bd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.746919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1a6af-19bb-4666-b9dd-e6a8fff9bd5c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.747416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1a6af-19bb-4666-b9dd-e6a8fff9bd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.747902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1a6af-19bb-4666-b9dd-e6a8fff9bd5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.764445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1a6af-19bb-4666-b9dd-e6a8fff9bd5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-f7a1a6af-19bb-4666-b9dd-e6a8fff9bd5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:31.770904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:31.771874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:31.772694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:31.774920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 129/282] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:31.781202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:31.907333 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b024f0-a7b5-4130-942f-b7bfae23362b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:31.926713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:31.947554 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:31.948186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:31.948469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:31.948748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:31.949752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:31.950059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:31.951427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:31.961449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 154/283] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:31.970580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1a6af-19bb-4666-b9dd-e6a8fff9bd5c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:32.093718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1a6af-19bb-4666-b9dd-e6a8fff9bd5c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:32.138586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cf7d7-8ce1-4d3e-9380-96a91d120adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:32.139232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cf7d7-8ce1-4d3e-9380-96a91d120adb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:32.139508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cf7d7-8ce1-4d3e-9380-96a91d120adb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:32.139779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cf7d7-8ce1-4d3e-9380-96a91d120adb None admin] PATH_INFO: `/v3/users/eca95181948046ac843318606f071c37` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:32.146032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cf7d7-8ce1-4d3e-9380-96a91d120adb None admin] RBAC: Authorizing `identity:get_user(user_id=eca95181948046ac843318606f071c37)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:32.147498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cf7d7-8ce1-4d3e-9380-96a91d120adb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:32.149117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 130/284] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:31 2022] GET /identity/v3/users/eca95181948046ac843318606f071c37 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:32.154655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:32.302935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a4281-5c4d-4257-ae3b-d7da4bfcbf8b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:32.344922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b34e6a-da16-4343-937c-8645b21083f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:32.345976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b34e6a-da16-4343-937c-8645b21083f9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:32.346513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b34e6a-da16-4343-937c-8645b21083f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:32.347082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b34e6a-da16-4343-937c-8645b21083f9 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:32.348600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b34e6a-da16-4343-937c-8645b21083f9 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:32.350309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b34e6a-da16-4343-937c-8645b21083f9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:32.352171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 155/285] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:32 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:32.358131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cf7d7-8ce1-4d3e-9380-96a91d120adb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:32.417632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:32.519242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cf7d7-8ce1-4d3e-9380-96a91d120adb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:32.577891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd75f6-26de-4e70-aa2c-9dda2a488a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:32.578489 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd75f6-26de-4e70-aa2c-9dda2a488a96 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:32.578773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd75f6-26de-4e70-aa2c-9dda2a488a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:32.579048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd75f6-26de-4e70-aa2c-9dda2a488a96 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/users/eca95181948046ac843318606f071c37/roles/3cc2fe72d0a245238f0d8df6763755c9` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:32.580996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd75f6-26de-4e70-aa2c-9dda2a488a96 None admin] RBAC: Authorizing `identity:create_grant(project_id=0fe9b2593d1947c19cb976bc9c6e51f1, user_id=eca95181948046ac843318606f071c37, role_id=3cc2fe72d0a245238f0d8df6763755c9)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:32.583076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd75f6-26de-4e70-aa2c-9dda2a488a96 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:32.606936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 131/286] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:23:32 2022] PUT /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/users/eca95181948046ac843318606f071c37/roles/3cc2fe72d0a245238f0d8df6763755c9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:32.805850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6e12-2058-4fd0-a7f6-5d1aef1750c0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:32.806537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6e12-2058-4fd0-a7f6-5d1aef1750c0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:32.807019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6e12-2058-4fd0-a7f6-5d1aef1750c0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:32.808980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 156/287] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:32.814333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14155f0c-ccf3-4f3d-ab47-467ed678fa19 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:32.814623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14155f0c-ccf3-4f3d-ab47-467ed678fa19 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:32.814902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14155f0c-ccf3-4f3d-ab47-467ed678fa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:32.842497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-14155f0c-ccf3-4f3d-ab47-467ed678fa19 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:32.843612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-14155f0c-ccf3-4f3d-ab47-467ed678fa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:32.929368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 132/288] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:32.957012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e6e12-2058-4fd0-a7f6-5d1aef1750c0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:33.022630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:33.106427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e6e12-2058-4fd0-a7f6-5d1aef1750c0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:33.155824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b025c97-ee15-42c7-b545-bcb7f6fb980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:33.156490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b025c97-ee15-42c7-b545-bcb7f6fb980c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:33.156733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b025c97-ee15-42c7-b545-bcb7f6fb980c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:33.157003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b025c97-ee15-42c7-b545-bcb7f6fb980c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:33.173375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b025c97-ee15-42c7-b545-bcb7f6fb980c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5b025c97-ee15-42c7-b545-bcb7f6fb980c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:33.182150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:33.183051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:33.183849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:33.187821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 157/289] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:33.189100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14155f0c-ccf3-4f3d-ab47-467ed678fa19 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:33.231966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:33.336292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14155f0c-ccf3-4f3d-ab47-467ed678fa19 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:33.372973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:33.373550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:33.373833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:33.374106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:33.374826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:33.375130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:33.376518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:33.389210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 133/290] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:33.395351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b025c97-ee15-42c7-b545-bcb7f6fb980c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:33.558452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b025c97-ee15-42c7-b545-bcb7f6fb980c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:33.572525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:33.616037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1a232-3526-4e5f-8a85-478031fb0ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:33.616689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1a232-3526-4e5f-8a85-478031fb0ebf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:33.616969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1a232-3526-4e5f-8a85-478031fb0ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:33.617245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1a232-3526-4e5f-8a85-478031fb0ebf None admin] PATH_INFO: `/v3/users/7cfe0d584975405c97afe5ba3302b7f0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:33.618475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1a232-3526-4e5f-8a85-478031fb0ebf None admin] RBAC: Authorizing `identity:get_user(user_id=7cfe0d584975405c97afe5ba3302b7f0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:33.619920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1a232-3526-4e5f-8a85-478031fb0ebf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:33.621889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 158/291] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:33 2022] GET /identity/v3/users/7cfe0d584975405c97afe5ba3302b7f0 => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:33.627109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:33.789789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:33.804086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8970d5-9e4b-4cf0-b495-90c3f86f79ed None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:33.863421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce88254-19e5-4bf4-9d6e-0cba22aa5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:33.864047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce88254-19e5-4bf4-9d6e-0cba22aa5643 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:33.864329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce88254-19e5-4bf4-9d6e-0cba22aa5643 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:33.864601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce88254-19e5-4bf4-9d6e-0cba22aa5643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:33.984698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce88254-19e5-4bf4-9d6e-0cba22aa5643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:33.986204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce88254-19e5-4bf4-9d6e-0cba22aa5643 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:34.039705 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9789af-9a94-49cf-a485-a6dd74c5adac None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.040022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9789af-9a94-49cf-a485-a6dd74c5adac None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.040252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9789af-9a94-49cf-a485-a6dd74c5adac None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.042740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 159/292] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:34.048149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc30c-e4b0-4d74-80f2-1c73bee232a2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.048432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc30c-e4b0-4d74-80f2-1c73bee232a2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.048709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc30c-e4b0-4d74-80f2-1c73bee232a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.071938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-202fc30c-e4b0-4d74-80f2-1c73bee232a2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:34.073567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-202fc30c-e4b0-4d74-80f2-1c73bee232a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:34.141217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:34.155769 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 160/293] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:34.182966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fc30c-e4b0-4d74-80f2-1c73bee232a2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:34.185021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 134/294] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:34.190364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce88254-19e5-4bf4-9d6e-0cba22aa5643 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:34.299429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fc30c-e4b0-4d74-80f2-1c73bee232a2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:34.309659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:34.350386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acc726-f1f8-40b4-a03e-4ef8d0c5ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:34.351046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc726-f1f8-40b4-a03e-4ef8d0c5ec10 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.351547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc726-f1f8-40b4-a03e-4ef8d0c5ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.351831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc726-f1f8-40b4-a03e-4ef8d0c5ec10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.355676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce88254-19e5-4bf4-9d6e-0cba22aa5643 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:34.365202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acc726-f1f8-40b4-a03e-4ef8d0c5ec10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e0acc726-f1f8-40b4-a03e-4ef8d0c5ec10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:34.372513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:34.373432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:34.374220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:34.377396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 161/295] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:34.379011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9a72-30d1-4c25-a289-dc07f67d2884 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.379292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9a72-30d1-4c25-a289-dc07f67d2884 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.379566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9a72-30d1-4c25-a289-dc07f67d2884 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.380968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 162/296] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:34.382649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d9a72-30d1-4c25-a289-dc07f67d2884 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:34.408910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211850a7-28ba-4211-8cc2-60dda45be534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:34.411046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211850a7-28ba-4211-8cc2-60dda45be534 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.411563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211850a7-28ba-4211-8cc2-60dda45be534 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.412294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211850a7-28ba-4211-8cc2-60dda45be534 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.413968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211850a7-28ba-4211-8cc2-60dda45be534 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:34.416555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211850a7-28ba-4211-8cc2-60dda45be534 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:34.419621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 135/297] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:34 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:34.421680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3600b-4194-479e-a23b-393d77efd3db None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.422363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3600b-4194-479e-a23b-393d77efd3db None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.422868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3600b-4194-479e-a23b-393d77efd3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.444573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-04d3600b-4194-479e-a23b-393d77efd3db None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:34.446780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-04d3600b-4194-479e-a23b-393d77efd3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:34.503598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d9a72-30d1-4c25-a289-dc07f67d2884 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:34.504421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:34.523240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 136/298] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:34.523888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3600b-4194-479e-a23b-393d77efd3db None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:34.553339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d821f37-66b3-4c03-bba0-3cf94d24c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:34.554215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821f37-66b3-4c03-bba0-3cf94d24c55f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.554672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821f37-66b3-4c03-bba0-3cf94d24c55f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.555156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821f37-66b3-4c03-bba0-3cf94d24c55f None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.555976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d821f37-66b3-4c03-bba0-3cf94d24c55f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:34.556340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d821f37-66b3-4c03-bba0-3cf94d24c55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:34.557803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d821f37-66b3-4c03-bba0-3cf94d24c55f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:34.571386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 163/299] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:34.571386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b17e3-5cca-4c27-99a3-d1f9aedff04e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.571386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b17e3-5cca-4c27-99a3-d1f9aedff04e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.571386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b17e3-5cca-4c27-99a3-d1f9aedff04e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.575190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 164/300] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:34.575854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b17e3-5cca-4c27-99a3-d1f9aedff04e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:34.605559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:34.639062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3600b-4194-479e-a23b-393d77efd3db None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:34.676582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:34.677253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.677566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.677890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.678944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] RBAC: Adding query filter params (role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1, user.id=7cfe0d584975405c97afe5ba3302b7f0) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:34.679258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:34.680747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:34.687054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 137/301] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:34 2022] GET /identity/v3/role_assignments?user.id=7cfe0d584975405c97afe5ba3302b7f0&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=551979dba2834cf397f2c386c2c64b2d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:34.688698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b17e3-5cca-4c27-99a3-d1f9aedff04e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:34.694311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:34.740635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9baa419-647a-4fd6-9190-f72bff874ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:34.742087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baa419-647a-4fd6-9190-f72bff874ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.742371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baa419-647a-4fd6-9190-f72bff874ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.742643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baa419-647a-4fd6-9190-f72bff874ea4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.743934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9baa419-647a-4fd6-9190-f72bff874ea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:34.745498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9baa419-647a-4fd6-9190-f72bff874ea4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:34.752219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 165/302] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:34.754243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d691156-ad54-4723-a50f-3c0306ca5e0f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.754618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d691156-ad54-4723-a50f-3c0306ca5e0f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.754897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d691156-ad54-4723-a50f-3c0306ca5e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.772275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:34.778923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6d691156-ad54-4723-a50f-3c0306ca5e0f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:34.781716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6d691156-ad54-4723-a50f-3c0306ca5e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:34.844532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7900f04-2e9d-4228-81a3-e71f2720d276 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:34.863659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 166/303] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:34.864482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d691156-ad54-4723-a50f-3c0306ca5e0f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:34.892807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1d7b-7851-4552-80f7-edcc629c95f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:34.893598 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1d7b-7851-4552-80f7-edcc629c95f6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:34.894533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1d7b-7851-4552-80f7-edcc629c95f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:34.894824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1d7b-7851-4552-80f7-edcc629c95f6 None admin] PATH_INFO: `/v3/users/35526e9278424f05b2ddcbc2ec84230f` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:34.896612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:34.914562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f1d7b-7851-4552-80f7-edcc629c95f6 None admin] RBAC: Authorizing `identity:get_user(user_id=35526e9278424f05b2ddcbc2ec84230f)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:34.916026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f1d7b-7851-4552-80f7-edcc629c95f6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:34.918825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 138/304] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:34 2022] GET /identity/v3/users/35526e9278424f05b2ddcbc2ec84230f => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:34.918825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1d7b-7851-4552-80f7-edcc629c95f6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:34.996259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d691156-ad54-4723-a50f-3c0306ca5e0f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.032152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567488c-726e-48cd-ad1f-c608ad82d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.032354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1d7b-7851-4552-80f7-edcc629c95f6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.032782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567488c-726e-48cd-ad1f-c608ad82d45b None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.033063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567488c-726e-48cd-ad1f-c608ad82d45b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.033359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567488c-726e-48cd-ad1f-c608ad82d45b None admin] PATH_INFO: `/v3/groups` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.034034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567488c-726e-48cd-ad1f-c608ad82d45b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:35.035516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567488c-726e-48cd-ad1f-c608ad82d45b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:35.039891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:35.054177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 167/305] 10.222.0.20 () {60 vars in 1134 bytes} [Thu Jul 28 06:23:34 2022] POST /identity/v3/groups => generated 229 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:35.055064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567488c-726e-48cd-ad1f-c608ad82d45b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:35.080993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10bd4-1925-4c10-a9fa-1fab0aea142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.082498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10bd4-1925-4c10-a9fa-1fab0aea142a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.083019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10bd4-1925-4c10-a9fa-1fab0aea142a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.083475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10bd4-1925-4c10-a9fa-1fab0aea142a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.096165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e10bd4-1925-4c10-a9fa-1fab0aea142a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b0e10bd4-1925-4c10-a9fa-1fab0aea142a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:35.101652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:35.102729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:35.103570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 139/306] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:35.108680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10bd4-1925-4c10-a9fa-1fab0aea142a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:35.196198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:35.218276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567488c-726e-48cd-ad1f-c608ad82d45b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.236650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10bd4-1925-4c10-a9fa-1fab0aea142a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.259364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca26d4-f02c-4fe8-9676-bd95484a329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.259364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca26d4-f02c-4fe8-9676-bd95484a329c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.259654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca26d4-f02c-4fe8-9676-bd95484a329c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.259728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca26d4-f02c-4fe8-9676-bd95484a329c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.260528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca26d4-f02c-4fe8-9676-bd95484a329c None admin] RBAC: Adding query filter params (role.id=3cc2fe72d0a245238f0d8df6763755c9, user.id=35526e9278424f05b2ddcbc2ec84230f, scope.system=all) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:35.260840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca26d4-f02c-4fe8-9676-bd95484a329c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:35.262312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca26d4-f02c-4fe8-9676-bd95484a329c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:35.267911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 168/307] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Jul 28 06:23:35 2022] GET /identity/v3/role_assignments?user.id=35526e9278424f05b2ddcbc2ec84230f&scope.system=all&role.id=3cc2fe72d0a245238f0d8df6763755c9 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:35.269772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f9238-5f39-4c1a-a1d8-4305d4d6ad72 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.270063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f9238-5f39-4c1a-a1d8-4305d4d6ad72 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.270340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f9238-5f39-4c1a-a1d8-4305d4d6ad72 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.271718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 169/308] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:35.276684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.277573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd719068-b397-49b2-ba8f-cb68b06980cb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.277728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.277863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd719068-b397-49b2-ba8f-cb68b06980cb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.278144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd719068-b397-49b2-ba8f-cb68b06980cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.278433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.278867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.279746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:35.280194 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:35.281878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:35.292272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 140/309] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:35.298711 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:35.303664 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fd719068-b397-49b2-ba8f-cb68b06980cb None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:35.304906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fd719068-b397-49b2-ba8f-cb68b06980cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:35.336729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:35.385782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 170/310] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:35.417356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd719068-b397-49b2-ba8f-cb68b06980cb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:35.438559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d2449-efce-4e8d-a8d3-5a863a07bde8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.482156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57498c8-9326-4bf9-b58c-7d822367f11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.482951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57498c8-9326-4bf9-b58c-7d822367f11a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.483348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57498c8-9326-4bf9-b58c-7d822367f11a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.483628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57498c8-9326-4bf9-b58c-7d822367f11a None admin] PATH_INFO: `/v3/users/0abb4fab27d04964a9811abe54f87081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.485071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57498c8-9326-4bf9-b58c-7d822367f11a None admin] RBAC: Authorizing `identity:get_user(user_id=0abb4fab27d04964a9811abe54f87081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:35.487015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57498c8-9326-4bf9-b58c-7d822367f11a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:35.489582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 141/311] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:35 2022] GET /identity/v3/users/0abb4fab27d04964a9811abe54f87081 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:35.494823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57498c8-9326-4bf9-b58c-7d822367f11a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:35.530787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:35.556728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd719068-b397-49b2-ba8f-cb68b06980cb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.595610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951f855-690a-485a-9c94-a6ee0455d43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.596688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951f855-690a-485a-9c94-a6ee0455d43c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.597401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951f855-690a-485a-9c94-a6ee0455d43c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.597877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951f855-690a-485a-9c94-a6ee0455d43c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.599993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:35.606016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57498c8-9326-4bf9-b58c-7d822367f11a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.610982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951f855-690a-485a-9c94-a6ee0455d43c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9951f855-690a-485a-9c94-a6ee0455d43c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:35.617424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:35.618336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:35.619139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:35.621914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 171/312] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:35.630030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951f855-690a-485a-9c94-a6ee0455d43c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:35.652195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b8cb-3a3a-469e-bebc-501255cb1e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.653185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b8cb-3a3a-469e-bebc-501255cb1e41 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.653707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b8cb-3a3a-469e-bebc-501255cb1e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.654180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b8cb-3a3a-469e-bebc-501255cb1e41 None admin] PATH_INFO: `/v3/system/users/0abb4fab27d04964a9811abe54f87081/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.655856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b8cb-3a3a-469e-bebc-501255cb1e41 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0abb4fab27d04964a9811abe54f87081, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:35.657471 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b8cb-3a3a-469e-bebc-501255cb1e41 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:35.678007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 142/313] 10.222.0.20 () {58 vars in 1334 bytes} [Thu Jul 28 06:23:35 2022] PUT /identity/v3/system/users/0abb4fab27d04964a9811abe54f87081/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:23:35.747531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951f855-690a-485a-9c94-a6ee0455d43c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.789276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.790037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.790321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.790631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.791363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:35.791712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:35.793168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:35.799774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:35.805691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 172/314] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:35.814484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b8cb-3a3a-469e-bebc-501255cb1e41 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:35.932907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b8cb-3a3a-469e-bebc-501255cb1e41 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:35.969845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cee0b-df02-4a87-8736-074a0984bbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:35.970729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cee0b-df02-4a87-8736-074a0984bbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:35.971183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cee0b-df02-4a87-8736-074a0984bbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:35.971603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cee0b-df02-4a87-8736-074a0984bbf4 None admin] PATH_INFO: `/v3/users/eca95181948046ac843318606f071c37` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:35.973636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cee0b-df02-4a87-8736-074a0984bbf4 None admin] RBAC: Authorizing `identity:get_user(user_id=eca95181948046ac843318606f071c37)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:35.975231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cee0b-df02-4a87-8736-074a0984bbf4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:35.981403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 143/315] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:35 2022] GET /identity/v3/users/eca95181948046ac843318606f071c37 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:35.986363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:36.100319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f0dea-e997-4d3d-b2c3-a0a70cc2da63 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:36.135502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881c4f-18dd-4af5-b751-a8260c696b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:36.136155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881c4f-18dd-4af5-b751-a8260c696b37 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:36.136434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881c4f-18dd-4af5-b751-a8260c696b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:36.136702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881c4f-18dd-4af5-b751-a8260c696b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:36.205177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:36.208814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12881c4f-18dd-4af5-b751-a8260c696b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:36.210291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12881c4f-18dd-4af5-b751-a8260c696b37 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:36.315032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 173/316] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:36.320959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cee0b-df02-4a87-8736-074a0984bbf4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:36.357241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:36.431748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cee0b-df02-4a87-8736-074a0984bbf4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:36.469599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c910e-da8c-44c2-a9ce-cdb285ff670a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:36.470171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c910e-da8c-44c2-a9ce-cdb285ff670a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:36.470450 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c910e-da8c-44c2-a9ce-cdb285ff670a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:36.470723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c910e-da8c-44c2-a9ce-cdb285ff670a None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:36.472504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c910e-da8c-44c2-a9ce-cdb285ff670a None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:36.473865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c910e-da8c-44c2-a9ce-cdb285ff670a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:36.475732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 144/317] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:36 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:36.481309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881c4f-18dd-4af5-b751-a8260c696b37 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:36.578869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:36.582425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881c4f-18dd-4af5-b751-a8260c696b37 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:36.617128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2e860-e230-4419-ba43-4a8401bc1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:36.617777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2e860-e230-4419-ba43-4a8401bc1e24 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:36.618056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2e860-e230-4419-ba43-4a8401bc1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:36.618329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2e860-e230-4419-ba43-4a8401bc1e24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:36.619105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2e860-e230-4419-ba43-4a8401bc1e24 None admin] RBAC: Adding query filter params (role.id=3cc2fe72d0a245238f0d8df6763755c9, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1, user.id=eca95181948046ac843318606f071c37) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:36.619413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2e860-e230-4419-ba43-4a8401bc1e24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:36.620823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2e860-e230-4419-ba43-4a8401bc1e24 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:36.627225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 174/318] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:23:36 2022] GET /identity/v3/role_assignments?user.id=eca95181948046ac843318606f071c37&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=3cc2fe72d0a245238f0d8df6763755c9 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:37.214798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bdb34-dc78-440f-b28c-fff2a9fd444c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.215172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bdb34-dc78-440f-b28c-fff2a9fd444c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.215375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bdb34-dc78-440f-b28c-fff2a9fd444c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.216772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 145/319] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:37.221846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a90af-e9f2-4539-8bc6-cb57ac18a2d7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.222310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a90af-e9f2-4539-8bc6-cb57ac18a2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.222391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a90af-e9f2-4539-8bc6-cb57ac18a2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.240858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-039a90af-e9f2-4539-8bc6-cb57ac18a2d7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:37.242074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-039a90af-e9f2-4539-8bc6-cb57ac18a2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:37.311269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 175/320] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:37.334424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bdb34-dc78-440f-b28c-fff2a9fd444c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:37.406389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d6bd5-71f1-455b-aef0-3c5f6e204ba7 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.406678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d6bd5-71f1-455b-aef0-3c5f6e204ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.406953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d6bd5-71f1-455b-aef0-3c5f6e204ba7 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.408320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 176/321] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:37.413567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4027a1f-aa43-4f6e-83b9-522e4e7e8b5a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.413930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4027a1f-aa43-4f6e-83b9-522e4e7e8b5a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.414211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4027a1f-aa43-4f6e-83b9-522e4e7e8b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.434962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f4027a1f-aa43-4f6e-83b9-522e4e7e8b5a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:37.436148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f4027a1f-aa43-4f6e-83b9-522e4e7e8b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:37.452553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:37.460424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bdb34-dc78-440f-b28c-fff2a9fd444c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:37.498342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7ac1c-8a79-4c30-9c85-30ce41804b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:37.498991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7ac1c-8a79-4c30-9c85-30ce41804b56 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.499276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7ac1c-8a79-4c30-9c85-30ce41804b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.499558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7ac1c-8a79-4c30-9c85-30ce41804b56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.501381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7ac1c-8a79-4c30-9c85-30ce41804b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:37.502826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7ac1c-8a79-4c30-9c85-30ce41804b56 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:37.505139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 146/322] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:37.510593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7ac1c-8a79-4c30-9c85-30ce41804b56 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:37.515941 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 177/323] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:37.537779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4027a1f-aa43-4f6e-83b9-522e4e7e8b5a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:37.562204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:37.619481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7ac1c-8a79-4c30-9c85-30ce41804b56 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:37.646987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4027a1f-aa43-4f6e-83b9-522e4e7e8b5a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:37.664574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28329e9-dd29-472e-b275-e67de0a5071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:37.665832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28329e9-dd29-472e-b275-e67de0a5071a None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.666297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28329e9-dd29-472e-b275-e67de0a5071a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.666747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28329e9-dd29-472e-b275-e67de0a5071a None admin] PATH_INFO: `/v3/groups` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.668069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28329e9-dd29-472e-b275-e67de0a5071a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:37.669988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28329e9-dd29-472e-b275-e67de0a5071a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:37.685740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0b71a-e7bf-4992-acc6-b47b5657aee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:37.686855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b71a-e7bf-4992-acc6-b47b5657aee5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.687366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b71a-e7bf-4992-acc6-b47b5657aee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.687783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b71a-e7bf-4992-acc6-b47b5657aee5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.688997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 147/324] 10.222.0.20 () {60 vars in 1134 bytes} [Thu Jul 28 06:23:37 2022] POST /identity/v3/groups => generated 226 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:23:37.710838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0b71a-e7bf-4992-acc6-b47b5657aee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-fab0b71a-e7bf-4992-acc6-b47b5657aee5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:37.716093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:37.717099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:37.723595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 178/325] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:37.725494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28329e9-dd29-472e-b275-e67de0a5071a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:37.755638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:37.841370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28329e9-dd29-472e-b275-e67de0a5071a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:37.841554 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749bee-cceb-4c7a-a361-35bce22f618f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.841780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749bee-cceb-4c7a-a361-35bce22f618f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.841960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74749bee-cceb-4c7a-a361-35bce22f618f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.843480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 179/326] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:37.849431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10254ee-213d-43cb-897d-58e10b7b08c8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.849721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10254ee-213d-43cb-897d-58e10b7b08c8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.850572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10254ee-213d-43cb-897d-58e10b7b08c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.871091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d10254ee-213d-43cb-897d-58e10b7b08c8 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:37.872293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d10254ee-213d-43cb-897d-58e10b7b08c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:37.882077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:37.882753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:37.882992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:37.883273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:37.883997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:37.884305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:37.885718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:37.895182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 148/327] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:37.901736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:37.952341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 180/328] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:37.976341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10254ee-213d-43cb-897d-58e10b7b08c8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:37.999437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:38.001944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:38.021843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b42b-bd54-468b-8ebe-5af53b6d42df None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:38.064800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3e64e-8b0f-4291-8379-88c43f810eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:38.068825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3e64e-8b0f-4291-8379-88c43f810eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:38.068825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3e64e-8b0f-4291-8379-88c43f810eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:38.068825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3e64e-8b0f-4291-8379-88c43f810eb1 None admin] PATH_INFO: `/v3/users/35526e9278424f05b2ddcbc2ec84230f` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:38.070856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3e64e-8b0f-4291-8379-88c43f810eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=35526e9278424f05b2ddcbc2ec84230f)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:38.071503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3e64e-8b0f-4291-8379-88c43f810eb1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:38.077659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 149/329] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:37 2022] GET /identity/v3/users/35526e9278424f05b2ddcbc2ec84230f => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:38.079076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3e64e-8b0f-4291-8379-88c43f810eb1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:38.103228 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10254ee-213d-43cb-897d-58e10b7b08c8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:38.155121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49745403-8bb0-4cd1-8290-e1410f48d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:38.156158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745403-8bb0-4cd1-8290-e1410f48d57f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:38.156665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745403-8bb0-4cd1-8290-e1410f48d57f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:38.157142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745403-8bb0-4cd1-8290-e1410f48d57f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:38.169440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49745403-8bb0-4cd1-8290-e1410f48d57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-49745403-8bb0-4cd1-8290-e1410f48d57f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:38.174785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:38.176202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:38.177031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:38.179365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 181/330] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:38.185039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49745403-8bb0-4cd1-8290-e1410f48d57f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:38.223407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3e64e-8b0f-4291-8379-88c43f810eb1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:38.266979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a82152-1ddc-465d-aa60-865f246e9b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:38.267330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a82152-1ddc-465d-aa60-865f246e9b3a None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:38.267607 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a82152-1ddc-465d-aa60-865f246e9b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:38.268006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a82152-1ddc-465d-aa60-865f246e9b3a None admin] PATH_INFO: `/v3/system/users/35526e9278424f05b2ddcbc2ec84230f/roles/3cc2fe72d0a245238f0d8df6763755c9` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:38.272005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a82152-1ddc-465d-aa60-865f246e9b3a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=35526e9278424f05b2ddcbc2ec84230f, role_id=3cc2fe72d0a245238f0d8df6763755c9)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:38.274006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a82152-1ddc-465d-aa60-865f246e9b3a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:38.288283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:38.293965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 150/331] 10.222.0.20 () {58 vars in 1334 bytes} [Thu Jul 28 06:23:38 2022] PUT /identity/v3/system/users/35526e9278424f05b2ddcbc2ec84230f/roles/3cc2fe72d0a245238f0d8df6763755c9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:23:38.322709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49745403-8bb0-4cd1-8290-e1410f48d57f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:38.365415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:38.366416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:38.366938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:38.367402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:38.368322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:38.368819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:38.370598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:38.382594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 182/332] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:38.388039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a82152-1ddc-465d-aa60-865f246e9b3a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:38.417698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:38.511172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a82152-1ddc-465d-aa60-865f246e9b3a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:38.547515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98273e3b-92c9-448b-872e-67dd081b6e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:38.548089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98273e3b-92c9-448b-872e-67dd081b6e5e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:38.548368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98273e3b-92c9-448b-872e-67dd081b6e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:38.548696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98273e3b-92c9-448b-872e-67dd081b6e5e None admin] PATH_INFO: `/v3/users/0abb4fab27d04964a9811abe54f87081` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:38.550099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98273e3b-92c9-448b-872e-67dd081b6e5e None admin] RBAC: Authorizing `identity:get_user(user_id=0abb4fab27d04964a9811abe54f87081)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:38.552102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98273e3b-92c9-448b-872e-67dd081b6e5e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:38.553781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 151/333] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:38 2022] GET /identity/v3/users/0abb4fab27d04964a9811abe54f87081 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:38.559650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:38.692247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99523715-6dbd-4d42-b1e9-f12cc27ed3ec None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:38.733895 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab61b0b-9217-470c-8e4b-a37718cadac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:38.734606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab61b0b-9217-470c-8e4b-a37718cadac2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:38.734885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab61b0b-9217-470c-8e4b-a37718cadac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:38.735160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab61b0b-9217-470c-8e4b-a37718cadac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:38.735944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab61b0b-9217-470c-8e4b-a37718cadac2 None admin] RBAC: Adding query filter params (role.id=551979dba2834cf397f2c386c2c64b2d, user.id=0abb4fab27d04964a9811abe54f87081, scope.system=all) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:38.736250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab61b0b-9217-470c-8e4b-a37718cadac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:38.738377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab61b0b-9217-470c-8e4b-a37718cadac2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:38.746565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:38.759279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 183/334] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Jul 28 06:23:38 2022] GET /identity/v3/role_assignments?user.id=0abb4fab27d04964a9811abe54f87081&scope.system=all&role.id=551979dba2834cf397f2c386c2c64b2d => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:40.116393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b6fd-884d-4501-b5d0-cfce35b2bf3a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:40.116905 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b6fd-884d-4501-b5d0-cfce35b2bf3a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:40.116980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b6fd-884d-4501-b5d0-cfce35b2bf3a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:40.118692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 152/335] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:40.123616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d609319-877b-47b1-8860-1049cbd3d567 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:40.124026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d609319-877b-47b1-8860-1049cbd3d567 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:40.124167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d609319-877b-47b1-8860-1049cbd3d567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:40.154325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2d609319-877b-47b1-8860-1049cbd3d567 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:40.155526 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2d609319-877b-47b1-8860-1049cbd3d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:40.235824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 184/336] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:40.276015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1b6fd-884d-4501-b5d0-cfce35b2bf3a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:40.392800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:40.412438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1b6fd-884d-4501-b5d0-cfce35b2bf3a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:40.453183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996897c-c604-4491-9dfb-28303a94b0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:40.453860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996897c-c604-4491-9dfb-28303a94b0ea None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:40.454141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996897c-c604-4491-9dfb-28303a94b0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:40.454867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996897c-c604-4491-9dfb-28303a94b0ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:40.469555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996897c-c604-4491-9dfb-28303a94b0ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5996897c-c604-4491-9dfb-28303a94b0ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:40.475163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:40.477345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:40.479331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 153/337] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:40.485689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d609319-877b-47b1-8860-1049cbd3d567 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:40.657363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d609319-877b-47b1-8860-1049cbd3d567 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:40.686678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:40.703175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a12a51-4cfa-478a-acf2-de1ab9b00eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:40.703856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12a51-4cfa-478a-acf2-de1ab9b00eea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:40.704141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12a51-4cfa-478a-acf2-de1ab9b00eea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:40.704415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12a51-4cfa-478a-acf2-de1ab9b00eea None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:40.705134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a12a51-4cfa-478a-acf2-de1ab9b00eea None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:40.705482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a12a51-4cfa-478a-acf2-de1ab9b00eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:40.707267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a12a51-4cfa-478a-acf2-de1ab9b00eea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:40.716169 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 185/338] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:40.722208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996897c-c604-4491-9dfb-28303a94b0ea None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:40.832754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7459-1d03-459e-9f3e-813e422f3297 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:40.833034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7459-1d03-459e-9f3e-813e422f3297 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:40.833525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7459-1d03-459e-9f3e-813e422f3297 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:40.835221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 186/339] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:40.840527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fb6a-df2c-4f09-bef2-cae612b9be96 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:40.841012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fb6a-df2c-4f09-bef2-cae612b9be96 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:40.841471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fb6a-df2c-4f09-bef2-cae612b9be96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:40.865277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996897c-c604-4491-9dfb-28303a94b0ea None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:40.878395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d679fb6a-df2c-4f09-bef2-cae612b9be96 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:40.879597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d679fb6a-df2c-4f09-bef2-cae612b9be96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:40.926169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213184cb-bf6b-467b-80fd-7f271aa15ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:40.927139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213184cb-bf6b-467b-80fd-7f271aa15ddd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:40.927617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213184cb-bf6b-467b-80fd-7f271aa15ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:40.928081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213184cb-bf6b-467b-80fd-7f271aa15ddd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:40.963472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:40.963788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 187/340] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:41.000266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923f99-947c-4c2d-aaee-800c30ebaef3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.000570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923f99-947c-4c2d-aaee-800c30ebaef3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.000814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923f99-947c-4c2d-aaee-800c30ebaef3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.002433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 188/341] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:41.010644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b923f99-947c-4c2d-aaee-800c30ebaef3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:41.026545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213184cb-bf6b-467b-80fd-7f271aa15ddd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:41.028092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213184cb-bf6b-467b-80fd-7f271aa15ddd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:41.158603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 154/342] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:41.163487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9c299-bccf-43cd-b210-132fc5522203 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.163778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9c299-bccf-43cd-b210-132fc5522203 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.164058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9c299-bccf-43cd-b210-132fc5522203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.191380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bbf9c299-bccf-43cd-b210-132fc5522203 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:41.193236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bbf9c299-bccf-43cd-b210-132fc5522203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:41.211067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b923f99-947c-4c2d-aaee-800c30ebaef3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:41.234987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:41.242080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:41.263470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aac75b-ff0b-4b2c-80ff-98aff0cdc902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:41.264138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aac75b-ff0b-4b2c-80ff-98aff0cdc902 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.264567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aac75b-ff0b-4b2c-80ff-98aff0cdc902 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.264842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aac75b-ff0b-4b2c-80ff-98aff0cdc902 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.269555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 155/343] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:41.270466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9c299-bccf-43cd-b210-132fc5522203 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:41.282345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aac75b-ff0b-4b2c-80ff-98aff0cdc902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-16aac75b-ff0b-4b2c-80ff-98aff0cdc902 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:41.291341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:41.292318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:41.293138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:41.293772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 189/344] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:41.294912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd23c4-af0c-4984-9ba2-903c4c9cccbe None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.295203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd23c4-af0c-4984-9ba2-903c4c9cccbe None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.295483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd23c4-af0c-4984-9ba2-903c4c9cccbe None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.296983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 190/345] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:41.302168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd23c4-af0c-4984-9ba2-903c4c9cccbe None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:41.396333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9c299-bccf-43cd-b210-132fc5522203 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:41.430104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd23c4-af0c-4984-9ba2-903c4c9cccbe None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:41.434418 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366044cf-ad6a-4923-a984-dee9698ce796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:41.435361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366044cf-ad6a-4923-a984-dee9698ce796 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.435843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366044cf-ad6a-4923-a984-dee9698ce796 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.436344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366044cf-ad6a-4923-a984-dee9698ce796 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.437767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366044cf-ad6a-4923-a984-dee9698ce796 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:41.439438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366044cf-ad6a-4923-a984-dee9698ce796 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:41.442096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 156/346] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:41 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:41.443510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983d80e-eb2c-4a45-ae30-0397c2142936 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.444011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983d80e-eb2c-4a45-ae30-0397c2142936 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.444476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983d80e-eb2c-4a45-ae30-0397c2142936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.470269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2983d80e-eb2c-4a45-ae30-0397c2142936 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:41.471902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2983d80e-eb2c-4a45-ae30-0397c2142936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:41.474175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:41.474806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.475094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.475366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.476375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:41.476686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:41.478090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:41.486890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 191/347] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:41.488406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:41.521945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:41.536748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:41.555229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 157/348] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:41.556411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983d80e-eb2c-4a45-ae30-0397c2142936 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:41.624527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1cd20-d0e8-4994-9b40-e14b826d26bd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:41.670261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae1d67-82cf-4505-9e27-6a56ed054073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:41.670918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae1d67-82cf-4505-9e27-6a56ed054073 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.671205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae1d67-82cf-4505-9e27-6a56ed054073 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.671482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae1d67-82cf-4505-9e27-6a56ed054073 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.687297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae1d67-82cf-4505-9e27-6a56ed054073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-84ae1d67-82cf-4505-9e27-6a56ed054073 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:41.696341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:41.697685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 192/349] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:41.698544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae1d67-82cf-4505-9e27-6a56ed054073 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:41.703846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983d80e-eb2c-4a45-ae30-0397c2142936 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:41.754951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d150ff-1f78-4d52-a80c-e1ae8898cb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:41.755987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d150ff-1f78-4d52-a80c-e1ae8898cb06 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.756257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d150ff-1f78-4d52-a80c-e1ae8898cb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.756531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d150ff-1f78-4d52-a80c-e1ae8898cb06 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.769034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d150ff-1f78-4d52-a80c-e1ae8898cb06 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:41.771759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d150ff-1f78-4d52-a80c-e1ae8898cb06 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:41.775382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 158/350] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:41 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:41.776816 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d150ff-1f78-4d52-a80c-e1ae8898cb06 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:41.830118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:41.839298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae1d67-82cf-4505-9e27-6a56ed054073 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:41.878986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:41.891349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe5f4a-4451-43ba-ab07-6e15019cbbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:41.894832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe5f4a-4451-43ba-ab07-6e15019cbbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.895127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe5f4a-4451-43ba-ab07-6e15019cbbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.895400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe5f4a-4451-43ba-ab07-6e15019cbbf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.923382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d150ff-1f78-4d52-a80c-e1ae8898cb06 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:41.970335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba2f7f-45f7-40ee-8b0f-4cc6724f7f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:41.971020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2f7f-45f7-40ee-8b0f-4cc6724f7f92 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:41.971311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2f7f-45f7-40ee-8b0f-4cc6724f7f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:41.972009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2f7f-45f7-40ee-8b0f-4cc6724f7f92 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:41.980401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe5f4a-4451-43ba-ab07-6e15019cbbf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:41.982856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe5f4a-4451-43ba-ab07-6e15019cbbf5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:41.985344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba2f7f-45f7-40ee-8b0f-4cc6724f7f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-07ba2f7f-45f7-40ee-8b0f-4cc6724f7f92 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:41.991773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:41.992743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:41.993648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 159/351] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:41.994355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba2f7f-45f7-40ee-8b0f-4cc6724f7f92 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.099250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:42.100454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:42.103590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba2f7f-45f7-40ee-8b0f-4cc6724f7f92 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:42.106113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 193/352] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:42.107470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe5f4a-4451-43ba-ab07-6e15019cbbf5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.141663 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:42.142316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.142597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.142868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.143630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:42.144005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:42.146945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:42.154650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 160/353] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:42.155582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.229010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe5f4a-4451-43ba-ab07-6e15019cbbf5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:42.264799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7adc22-8c24-4c45-a750-eb92cd65b5d6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:42.267808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:42.268736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.269407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.269901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.270889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] RBAC: Adding query filter params (group.id=b6169fd52d40419b87370474c1e963b0, role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:42.271402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:42.273029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:42.279087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 194/354] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:42 2022] GET /identity/v3/role_assignments?group.id=b6169fd52d40419b87370474c1e963b0&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=551979dba2834cf397f2c386c2c64b2d => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:42.280125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.304617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843732-1545-47be-bda6-581346627842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:42.305263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843732-1545-47be-bda6-581346627842 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.305572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843732-1545-47be-bda6-581346627842 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.305845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843732-1545-47be-bda6-581346627842 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.306591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b843732-1545-47be-bda6-581346627842 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:42.306896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b843732-1545-47be-bda6-581346627842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:42.308327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b843732-1545-47be-bda6-581346627842 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:42.318997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 161/355] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:42.319759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843732-1545-47be-bda6-581346627842 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.349720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:42.369659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:42.417282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28fd9c-4cf7-4a72-9e4d-3b6abbea13fe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:42.435936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843732-1545-47be-bda6-581346627842 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:42.462449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedd1c7-0d40-48b9-8de7-c429170b3c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:42.463154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedd1c7-0d40-48b9-8de7-c429170b3c58 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.463442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedd1c7-0d40-48b9-8de7-c429170b3c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.463718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedd1c7-0d40-48b9-8de7-c429170b3c58 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.466337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedd1c7-0d40-48b9-8de7-c429170b3c58 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:42.467814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedd1c7-0d40-48b9-8de7-c429170b3c58 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:42.470177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 195/356] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:42 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:42.471403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad0bd4-f17b-4e80-8cd7-a01d0db4bfc5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.472448 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad0bd4-f17b-4e80-8cd7-a01d0db4bfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.472824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad0bd4-f17b-4e80-8cd7-a01d0db4bfc5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.477684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a172457-6e59-434d-88d9-b5bd4e273cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:42.478324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a172457-6e59-434d-88d9-b5bd4e273cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.478606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a172457-6e59-434d-88d9-b5bd4e273cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.478879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a172457-6e59-434d-88d9-b5bd4e273cf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.479354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 196/357] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:42.480240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad0bd4-f17b-4e80-8cd7-a01d0db4bfc5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.558640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a172457-6e59-434d-88d9-b5bd4e273cf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:42.560172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a172457-6e59-434d-88d9-b5bd4e273cf3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:42.589453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:42.623803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad0bd4-f17b-4e80-8cd7-a01d0db4bfc5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:42.666883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:42.678869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b768c-3623-4657-86c0-dcaa3c98cec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:42.679935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b768c-3623-4657-86c0-dcaa3c98cec9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.680476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b768c-3623-4657-86c0-dcaa3c98cec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.680897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b768c-3623-4657-86c0-dcaa3c98cec9 None admin] PATH_INFO: `/v3/users/35526e9278424f05b2ddcbc2ec84230f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.683033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b768c-3623-4657-86c0-dcaa3c98cec9 None admin] RBAC: Authorizing `identity:get_user(user_id=35526e9278424f05b2ddcbc2ec84230f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:42.684941 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b768c-3623-4657-86c0-dcaa3c98cec9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:42.688115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 197/358] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:23:42 2022] GET /identity/v3/users/35526e9278424f05b2ddcbc2ec84230f => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:42.689195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b768c-3623-4657-86c0-dcaa3c98cec9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.691062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 162/359] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:42.696884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7afc-a5be-4006-b77b-257f1356348e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.697375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7afc-a5be-4006-b77b-257f1356348e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.697769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7afc-a5be-4006-b77b-257f1356348e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.718608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b57c7afc-a5be-4006-b77b-257f1356348e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:42.719810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b57c7afc-a5be-4006-b77b-257f1356348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:42.812926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 163/360] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:42.813972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ac2a3-f9e0-47a4-b8a8-428f91cd57bf None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.814265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ac2a3-f9e0-47a4-b8a8-428f91cd57bf None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.814541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ac2a3-f9e0-47a4-b8a8-428f91cd57bf None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.816147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 164/361] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:42.816742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ac2a3-f9e0-47a4-b8a8-428f91cd57bf None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.826719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b768c-3623-4657-86c0-dcaa3c98cec9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:42.870100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687544f-7584-40e8-94c2-6dcc92fae1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:42.870746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687544f-7584-40e8-94c2-6dcc92fae1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:42.871026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687544f-7584-40e8-94c2-6dcc92fae1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:42.871300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687544f-7584-40e8-94c2-6dcc92fae1d9 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:42.872928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687544f-7584-40e8-94c2-6dcc92fae1d9 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:42.874428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687544f-7584-40e8-94c2-6dcc92fae1d9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:42.876391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 198/362] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:42 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:42.877153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687544f-7584-40e8-94c2-6dcc92fae1d9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:42.906217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:42.967207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:42.967998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ac2a3-f9e0-47a4-b8a8-428f91cd57bf None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.011904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c882ae-4597-4e5e-96e0-67933262949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.012293 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c882ae-4597-4e5e-96e0-67933262949c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.012572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c882ae-4597-4e5e-96e0-67933262949c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.012667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687544f-7584-40e8-94c2-6dcc92fae1d9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.012845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c882ae-4597-4e5e-96e0-67933262949c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.013662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c882ae-4597-4e5e-96e0-67933262949c None admin] RBAC: Adding query filter params (role.id=3cc2fe72d0a245238f0d8df6763755c9, user.id=35526e9278424f05b2ddcbc2ec84230f, scope.system=all) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:43.013972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c882ae-4597-4e5e-96e0-67933262949c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:43.015523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c882ae-4597-4e5e-96e0-67933262949c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:43.029193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 165/363] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Jul 28 06:23:42 2022] GET /identity/v3/role_assignments?user.id=35526e9278424f05b2ddcbc2ec84230f&scope.system=all&role.id=3cc2fe72d0a245238f0d8df6763755c9 => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:43.030306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbea33-f70e-46b3-9343-5870910b36ac None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.030591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbea33-f70e-46b3-9343-5870910b36ac None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.030864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbea33-f70e-46b3-9343-5870910b36ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.054238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-45cbea33-f70e-46b3-9343-5870910b36ac None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:43.055455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-45cbea33-f70e-46b3-9343-5870910b36ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:43.070212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9482f44-fa6c-4b6a-b415-5b7421ff929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.070791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9482f44-fa6c-4b6a-b415-5b7421ff929b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.071067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9482f44-fa6c-4b6a-b415-5b7421ff929b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.071332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9482f44-fa6c-4b6a-b415-5b7421ff929b None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.072667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9482f44-fa6c-4b6a-b415-5b7421ff929b None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:43.074414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9482f44-fa6c-4b6a-b415-5b7421ff929b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:43.076471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 199/364] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:42 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:43.077277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9482f44-fa6c-4b6a-b415-5b7421ff929b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.147490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 166/365] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:43.148409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbea33-f70e-46b3-9343-5870910b36ac None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.204327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:43.211301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9482f44-fa6c-4b6a-b415-5b7421ff929b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.264868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee5666-a4cc-4a12-805b-e6c0e65a0b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.265650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee5666-a4cc-4a12-805b-e6c0e65a0b6a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.266070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee5666-a4cc-4a12-805b-e6c0e65a0b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.266342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee5666-a4cc-4a12-805b-e6c0e65a0b6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.269269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:43.284483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee5666-a4cc-4a12-805b-e6c0e65a0b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:43.286750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbea33-f70e-46b3-9343-5870910b36ac None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-31ee5666-a4cc-4a12-805b-e6c0e65a0b6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:43.290284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:43.291193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:43.291994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:43.295470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 200/366] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:43.296359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee5666-a4cc-4a12-805b-e6c0e65a0b6a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.337890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.338535 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.338729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.338986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.339782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] RBAC: Adding query filter params (group.id=b6169fd52d40419b87370474c1e963b0, role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:43.340094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:43.343663 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:43.348830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 167/367] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/role_assignments?group.id=b6169fd52d40419b87370474c1e963b0&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=74f371799fd245bf987e798f5c342e6a => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:43.349819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.472751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee5666-a4cc-4a12-805b-e6c0e65a0b6a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.513488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f838e82-c6aa-49b1-bf09-12342ed21a1f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.516047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:43.519855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fb86e-0bc2-4de2-aaa1-198e36a28b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.520857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb86e-0bc2-4de2-aaa1-198e36a28b12 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.521367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb86e-0bc2-4de2-aaa1-198e36a28b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.521859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb86e-0bc2-4de2-aaa1-198e36a28b12 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.523375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fb86e-0bc2-4de2-aaa1-198e36a28b12 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:43.525248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fb86e-0bc2-4de2-aaa1-198e36a28b12 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:43.527410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 201/368] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:43.528421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fb86e-0bc2-4de2-aaa1-198e36a28b12 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.552446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd9061-6073-4dfa-8048-89e60031eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.553106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd9061-6073-4dfa-8048-89e60031eee5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.553420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd9061-6073-4dfa-8048-89e60031eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.553697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd9061-6073-4dfa-8048-89e60031eee5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.564440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd9061-6073-4dfa-8048-89e60031eee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-82cd9061-6073-4dfa-8048-89e60031eee5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:43.570184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:43.571104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:43.571914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:43.576120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 168/369] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:43.577036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd9061-6073-4dfa-8048-89e60031eee5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.615039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:43.662957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fb86e-0bc2-4de2-aaa1-198e36a28b12 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.688620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd9061-6073-4dfa-8048-89e60031eee5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.702834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.703782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.704230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.704712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.705614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:43.706747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:43.708430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:43.717904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 202/370] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:43.719873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.727071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.727708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.727992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.728269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.729068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] RBAC: Adding query filter params (group.id=b6169fd52d40419b87370474c1e963b0, role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:43.729407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:43.730843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:43.736587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 169/371] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/role_assignments?group.id=b6169fd52d40419b87370474c1e963b0&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=551979dba2834cf397f2c386c2c64b2d => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:43.738886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.771957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:43.844496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de8c1e-ff9f-441b-b753-e63f9294e7b6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.872391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:43.885387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.886383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.886805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f4979-9eb1-43d9-a090-641df6c54905 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:43.887243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.887716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:43.888777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:43.889321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:43.891001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:43.901130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 203/372] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:43.907494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:43.947608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41e4dc-6984-4d77-9abe-1c14e1ec3a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:43.949137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41e4dc-6984-4d77-9abe-1c14e1ec3a71 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:43.949137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41e4dc-6984-4d77-9abe-1c14e1ec3a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:43.949137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41e4dc-6984-4d77-9abe-1c14e1ec3a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.022922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29fd3-e8c0-4261-bedd-faf4ee682317 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:44.061949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ee238-6b58-4dcc-a2e0-19e816463675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:44.063015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ee238-6b58-4dcc-a2e0-19e816463675 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:44.063524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ee238-6b58-4dcc-a2e0-19e816463675 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:44.064025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ee238-6b58-4dcc-a2e0-19e816463675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.106406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41e4dc-6984-4d77-9abe-1c14e1ec3a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:44.107873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41e4dc-6984-4d77-9abe-1c14e1ec3a71 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:44.141741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:44.162399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ee238-6b58-4dcc-a2e0-19e816463675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:44.163860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ee238-6b58-4dcc-a2e0-19e816463675 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:44.166648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:44.305576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 170/373] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:44.324323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41e4dc-6984-4d77-9abe-1c14e1ec3a71 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:44.349630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 204/374] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:23:44.356683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ee238-6b58-4dcc-a2e0-19e816463675 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:44.500112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41e4dc-6984-4d77-9abe-1c14e1ec3a71 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:44.522925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:44.545565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:44.549321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ee238-6b58-4dcc-a2e0-19e816463675 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:44.563119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddde10a-cc8e-4271-9272-f1cc20b1ed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:44.563766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddde10a-cc8e-4271-9272-f1cc20b1ed58 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:44.564040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddde10a-cc8e-4271-9272-f1cc20b1ed58 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:44.564310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddde10a-cc8e-4271-9272-f1cc20b1ed58 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.566107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddde10a-cc8e-4271-9272-f1cc20b1ed58 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:44.567555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddde10a-cc8e-4271-9272-f1cc20b1ed58 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:44.569636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 171/375] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:44 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:44.577777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddde10a-cc8e-4271-9272-f1cc20b1ed58 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:44.623214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5ad4e-ed5f-4c57-abb3-301730162feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:44.623858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5ad4e-ed5f-4c57-abb3-301730162feb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:44.624137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5ad4e-ed5f-4c57-abb3-301730162feb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:44.624405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5ad4e-ed5f-4c57-abb3-301730162feb None admin] PATH_INFO: `/v3/projects/25016d71f554431cbc0d0222ca313083` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.625637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5ad4e-ed5f-4c57-abb3-301730162feb None admin] RBAC: Authorizing `identity:get_project(project_id=25016d71f554431cbc0d0222ca313083)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:44.627063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5ad4e-ed5f-4c57-abb3-301730162feb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:44.629766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 205/376] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:44 2022] GET /identity/v3/projects/25016d71f554431cbc0d0222ca313083 => generated 344 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:44.635873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5ad4e-ed5f-4c57-abb3-301730162feb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:44.723478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddde10a-cc8e-4271-9272-f1cc20b1ed58 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:44.766114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9e76c-0052-431c-bb98-f5a863bb23ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:44.766763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e76c-0052-431c-bb98-f5a863bb23ad None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:44.767040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e76c-0052-431c-bb98-f5a863bb23ad None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:44.767311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e76c-0052-431c-bb98-f5a863bb23ad None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.769832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9e76c-0052-431c-bb98-f5a863bb23ad None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:44.771281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9e76c-0052-431c-bb98-f5a863bb23ad None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:44.772892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 172/377] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:44 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:44.774050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73713d5-a631-4c70-bd1c-fad9b67ed7ba None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:44.774626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73713d5-a631-4c70-bd1c-fad9b67ed7ba None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:44.774900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73713d5-a631-4c70-bd1c-fad9b67ed7ba None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.776367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 173/378] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:44.780914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73713d5-a631-4c70-bd1c-fad9b67ed7ba None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:44.837279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5ad4e-ed5f-4c57-abb3-301730162feb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:44.866926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:44.878140 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72e094-bc03-4c38-ac30-b81220727ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:44.878790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72e094-bc03-4c38-ac30-b81220727ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:44.879066 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72e094-bc03-4c38-ac30-b81220727ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:44.879336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72e094-bc03-4c38-ac30-b81220727ee2 None admin] PATH_INFO: `/v3/groups/81d352431a674a24aee48ba4125db057` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.886479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72e094-bc03-4c38-ac30-b81220727ee2 None admin] RBAC: Authorizing `identity:get_group(group_id=81d352431a674a24aee48ba4125db057)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:44.887993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72e094-bc03-4c38-ac30-b81220727ee2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:44.890007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 206/379] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:44 2022] GET /identity/v3/groups/81d352431a674a24aee48ba4125db057 => generated 229 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:44.891335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803c2f7-712e-4296-8d3c-c1737fa777c6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:44.891605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803c2f7-712e-4296-8d3c-c1737fa777c6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:44.891881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803c2f7-712e-4296-8d3c-c1737fa777c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.911892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73713d5-a631-4c70-bd1c-fad9b67ed7ba None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:44.915994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d803c2f7-712e-4296-8d3c-c1737fa777c6 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:44.917283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d803c2f7-712e-4296-8d3c-c1737fa777c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:44.935072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:44.958000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:44.959245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:44.959544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:44.959820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:44.960671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] RBAC: Adding query filter params (group.id=b6169fd52d40419b87370474c1e963b0, role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:44.960982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:44.962555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:44.969994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 174/380] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:44 2022] GET /identity/v3/role_assignments?group.id=b6169fd52d40419b87370474c1e963b0&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=74f371799fd245bf987e798f5c342e6a => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:44.970864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:45.045344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 207/381] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:45.108109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a30e9-2874-4d8d-83d0-9ee64a3f2e64 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:45.133541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803c2f7-712e-4296-8d3c-c1737fa777c6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:45.139850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:45.150186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:45.150843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:45.151124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:45.151399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:45.152210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] RBAC: Adding query filter params (group.id=81d352431a674a24aee48ba4125db057, role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=25016d71f554431cbc0d0222ca313083) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:45.152518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:45.154008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:45.164051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 175/382] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:44 2022] GET /identity/v3/role_assignments?group.id=81d352431a674a24aee48ba4125db057&scope.project.id=25016d71f554431cbc0d0222ca313083&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:45.307017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803c2f7-712e-4296-8d3c-c1737fa777c6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:45.350303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad53e41-863b-4331-b1b6-60377337a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:45.351064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad53e41-863b-4331-b1b6-60377337a48a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:45.351247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad53e41-863b-4331-b1b6-60377337a48a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:45.351493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad53e41-863b-4331-b1b6-60377337a48a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:45.364934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad53e41-863b-4331-b1b6-60377337a48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5ad53e41-863b-4331-b1b6-60377337a48a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:45.369915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:45.372047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:45.374085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 208/383] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:45.381338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:45.514397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be89fa2-0429-409c-87c8-622b3ef4ad37 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:45.557567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:45.558111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:45.558395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:45.558669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:45.559388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:45.559691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:45.561067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:45.575292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 176/384] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:45.576584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad53e41-863b-4331-b1b6-60377337a48a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:45.597161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:45.730421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad53e41-863b-4331-b1b6-60377337a48a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:45.781257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f534d-3a9e-4723-80d8-d9caac8fdeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:45.781922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f534d-3a9e-4723-80d8-d9caac8fdeea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:45.782205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f534d-3a9e-4723-80d8-d9caac8fdeea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:45.782477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f534d-3a9e-4723-80d8-d9caac8fdeea None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:45.785993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f534d-3a9e-4723-80d8-d9caac8fdeea None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:45.787450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f534d-3a9e-4723-80d8-d9caac8fdeea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:45.789386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 209/385] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:45 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:45.795048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:45.855548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:45.906777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d669ed4-0ecf-41d0-9c35-4347d053e062 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:45.918167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22701aaa-1e86-41b1-b07c-544bff213338 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:45.918442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22701aaa-1e86-41b1-b07c-544bff213338 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:45.918720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22701aaa-1e86-41b1-b07c-544bff213338 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:45.920057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 210/386] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:45.928640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece16cf8-a1c8-4768-a9c4-1516688ba010 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:45.928640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece16cf8-a1c8-4768-a9c4-1516688ba010 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:45.928640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece16cf8-a1c8-4768-a9c4-1516688ba010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:45.951065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39cfee-da1e-4208-b3da-1bd85c88600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:45.951727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39cfee-da1e-4208-b3da-1bd85c88600d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:45.952083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39cfee-da1e-4208-b3da-1bd85c88600d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:45.952361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39cfee-da1e-4208-b3da-1bd85c88600d None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:45.958092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ece16cf8-a1c8-4768-a9c4-1516688ba010 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:45.958259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39cfee-da1e-4208-b3da-1bd85c88600d None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:45.961226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39cfee-da1e-4208-b3da-1bd85c88600d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:45.961362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 177/387] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:45 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:45.962732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ece16cf8-a1c8-4768-a9c4-1516688ba010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:45.967020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39cfee-da1e-4208-b3da-1bd85c88600d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:46.042555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 211/388] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:46.092671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece16cf8-a1c8-4768-a9c4-1516688ba010 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:46.106546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:46.136926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39cfee-da1e-4208-b3da-1bd85c88600d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:46.176465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:46.199614 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f3faa-ac06-40f8-accf-62f392deac5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:46.200297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3faa-ac06-40f8-accf-62f392deac5b None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:46.200583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3faa-ac06-40f8-accf-62f392deac5b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:46.200864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f3faa-ac06-40f8-accf-62f392deac5b None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/groups/b6169fd52d40419b87370474c1e963b0/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:46.207239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f3faa-ac06-40f8-accf-62f392deac5b None admin] RBAC: Authorizing `identity:create_grant(project_id=89d1fa86df9a40fe96f2a7554edbbaed, group_id=b6169fd52d40419b87370474c1e963b0, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:46.208696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f3faa-ac06-40f8-accf-62f392deac5b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:46.239989 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 178/389] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jul 28 06:23:45 2022] PUT /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/groups/b6169fd52d40419b87370474c1e963b0/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:46.256324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece16cf8-a1c8-4768-a9c4-1516688ba010 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:46.298700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca85b8-7f15-4604-a2ac-3eb2426b24e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:46.299422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca85b8-7f15-4604-a2ac-3eb2426b24e2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:46.299749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca85b8-7f15-4604-a2ac-3eb2426b24e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:46.300011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca85b8-7f15-4604-a2ac-3eb2426b24e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:46.312203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca85b8-7f15-4604-a2ac-3eb2426b24e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-07ca85b8-7f15-4604-a2ac-3eb2426b24e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:46.321860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:46.322811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:46.323607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 212/390] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:46.329002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f3faa-ac06-40f8-accf-62f392deac5b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:46.507769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f3faa-ac06-40f8-accf-62f392deac5b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:46.545275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5cdf1-18d6-461d-b597-6e477d1ccba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:46.545923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5cdf1-18d6-461d-b597-6e477d1ccba9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:46.546204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5cdf1-18d6-461d-b597-6e477d1ccba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:46.546577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5cdf1-18d6-461d-b597-6e477d1ccba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:46.547195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5cdf1-18d6-461d-b597-6e477d1ccba9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:46.547500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5cdf1-18d6-461d-b597-6e477d1ccba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:46.548855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5cdf1-18d6-461d-b597-6e477d1ccba9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:46.559940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 179/391] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:46.565861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca85b8-7f15-4604-a2ac-3eb2426b24e2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:46.639338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:46.663757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb7466-f798-4f01-878a-648fdfcc29bf None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:46.664078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb7466-f798-4f01-878a-648fdfcc29bf None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:46.664311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb7466-f798-4f01-878a-648fdfcc29bf None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:46.666139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 180/392] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:46.676804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adfb8b-7192-4380-9ddd-a8b1069fc776 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:46.677557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adfb8b-7192-4380-9ddd-a8b1069fc776 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:46.678030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adfb8b-7192-4380-9ddd-a8b1069fc776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:46.682654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca85b8-7f15-4604-a2ac-3eb2426b24e2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:46.711794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-16adfb8b-7192-4380-9ddd-a8b1069fc776 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:46.715865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-16adfb8b-7192-4380-9ddd-a8b1069fc776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:46.748830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03476d-f9df-4c49-9653-9586a9132dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:46.749844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03476d-f9df-4c49-9653-9586a9132dac None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:46.750348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03476d-f9df-4c49-9653-9586a9132dac None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:46.750823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03476d-f9df-4c49-9653-9586a9132dac None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:46.754210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03476d-f9df-4c49-9653-9586a9132dac None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:46.756825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03476d-f9df-4c49-9653-9586a9132dac None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:46.760164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 213/393] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:46 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:46.767138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03476d-f9df-4c49-9653-9586a9132dac None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:46.802463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:46.889438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 181/394] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:46.935882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03476d-f9df-4c49-9653-9586a9132dac None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:46.982951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ce5fd-0024-45b6-a103-31c698c45a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:46.983621 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ce5fd-0024-45b6-a103-31c698c45a0c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:46.983908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ce5fd-0024-45b6-a103-31c698c45a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:46.984189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ce5fd-0024-45b6-a103-31c698c45a0c None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:46.985413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ce5fd-0024-45b6-a103-31c698c45a0c None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:46.987784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ce5fd-0024-45b6-a103-31c698c45a0c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:46.989863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 214/395] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:46 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:46.995630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adfb8b-7192-4380-9ddd-a8b1069fc776 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:46.996155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ce5fd-0024-45b6-a103-31c698c45a0c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:47.001317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:47.114368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ce5fd-0024-45b6-a103-31c698c45a0c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:47.154321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334381-2cf9-41f7-ad3f-9f30d4141e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:47.155707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334381-2cf9-41f7-ad3f-9f30d4141e00 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.156388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334381-2cf9-41f7-ad3f-9f30d4141e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.157056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334381-2cf9-41f7-ad3f-9f30d4141e00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.175589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e334381-2cf9-41f7-ad3f-9f30d4141e00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3e334381-2cf9-41f7-ad3f-9f30d4141e00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:47.185001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:47.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:47.187061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:47.189788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 215/396] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:47.191967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adfb8b-7192-4380-9ddd-a8b1069fc776 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:47.197983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334381-2cf9-41f7-ad3f-9f30d4141e00 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:47.250094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d44952-655c-4d02-a174-17cd26b9a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:47.250691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d44952-655c-4d02-a174-17cd26b9a5ee None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.250975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d44952-655c-4d02-a174-17cd26b9a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.251245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d44952-655c-4d02-a174-17cd26b9a5ee None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/groups/b6169fd52d40419b87370474c1e963b0/roles/74f371799fd245bf987e798f5c342e6a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.254478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d44952-655c-4d02-a174-17cd26b9a5ee None admin] RBAC: Authorizing `identity:create_grant(project_id=89d1fa86df9a40fe96f2a7554edbbaed, group_id=b6169fd52d40419b87370474c1e963b0, role_id=74f371799fd245bf987e798f5c342e6a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:47.255980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d44952-655c-4d02-a174-17cd26b9a5ee None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:47.275479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:47.289025 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 182/397] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jul 28 06:23:46 2022] PUT /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed/groups/b6169fd52d40419b87370474c1e963b0/roles/74f371799fd245bf987e798f5c342e6a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:47.325243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334381-2cf9-41f7-ad3f-9f30d4141e00 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:47.365195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:47.365887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.366170 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.366446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.367188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:47.367498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:47.368909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:47.385582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 216/398] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:47.387100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d44952-655c-4d02-a174-17cd26b9a5ee None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:47.438053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:47.535938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d44952-655c-4d02-a174-17cd26b9a5ee None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:47.586175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e188d-23df-40b5-82fe-324b6cb2dcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:47.586950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e188d-23df-40b5-82fe-324b6cb2dcf5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.587153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e188d-23df-40b5-82fe-324b6cb2dcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.587384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e188d-23df-40b5-82fe-324b6cb2dcf5 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.588591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e188d-23df-40b5-82fe-324b6cb2dcf5 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:47.592345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e188d-23df-40b5-82fe-324b6cb2dcf5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:47.594424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 183/399] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:47 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:47.603128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:47.718452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a7408-1ed9-42da-ab15-00cd80dedae8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:47.743791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:47.746756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d9743-88b5-46c3-a190-28d6deba4a8b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.747089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d9743-88b5-46c3-a190-28d6deba4a8b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.747372 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d9743-88b5-46c3-a190-28d6deba4a8b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.750684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 184/400] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:47.755941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eee2ae-a265-427a-a150-b47d14a9e214 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.756224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eee2ae-a265-427a-a150-b47d14a9e214 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.756502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eee2ae-a265-427a-a150-b47d14a9e214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.762894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf3711-5ae5-4b6d-a9a3-c5e44c1bc003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:47.763559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3711-5ae5-4b6d-a9a3-c5e44c1bc003 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.763844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3711-5ae5-4b6d-a9a3-c5e44c1bc003 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.764121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf3711-5ae5-4b6d-a9a3-c5e44c1bc003 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.765357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf3711-5ae5-4b6d-a9a3-c5e44c1bc003 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:47.766783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf3711-5ae5-4b6d-a9a3-c5e44c1bc003 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:47.769121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 217/401] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:47 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:47.775062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf3711-5ae5-4b6d-a9a3-c5e44c1bc003 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:47.793879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-54eee2ae-a265-427a-a150-b47d14a9e214 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:47.795052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-54eee2ae-a265-427a-a150-b47d14a9e214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:47.899231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf3711-5ae5-4b6d-a9a3-c5e44c1bc003 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:47.946320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 185/402] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:47.947693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4e0a-0725-4c25-be6f-fe9ed501541a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.948010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4e0a-0725-4c25-be6f-fe9ed501541a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.948260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4e0a-0725-4c25-be6f-fe9ed501541a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.949603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3028990-0d4f-4b69-a1a2-dd7a8426e839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:47.950059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 186/403] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:47.953875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3028990-0d4f-4b69-a1a2-dd7a8426e839 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.954229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3028990-0d4f-4b69-a1a2-dd7a8426e839 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.954509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3028990-0d4f-4b69-a1a2-dd7a8426e839 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/groups/b6169fd52d40419b87370474c1e963b0/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.956397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370c08-f785-46be-a7c0-51f17ad3c07f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:47.956678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370c08-f785-46be-a7c0-51f17ad3c07f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:47.956952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370c08-f785-46be-a7c0-51f17ad3c07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:47.958616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3028990-0d4f-4b69-a1a2-dd7a8426e839 None admin] RBAC: Authorizing `identity:create_grant(project_id=0fe9b2593d1947c19cb976bc9c6e51f1, group_id=b6169fd52d40419b87370474c1e963b0, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:47.960391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3028990-0d4f-4b69-a1a2-dd7a8426e839 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:47.980654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 218/404] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jul 28 06:23:47 2022] PUT /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/groups/b6169fd52d40419b87370474c1e963b0/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:47.990168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a6370c08-f785-46be-a7c0-51f17ad3c07f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:47.991566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a6370c08-f785-46be-a7c0-51f17ad3c07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:48.004093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3028990-0d4f-4b69-a1a2-dd7a8426e839 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:48.039904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:48.077265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:48.114315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 187/405] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:48.136303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6370c08-f785-46be-a7c0-51f17ad3c07f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:48.164287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3028990-0d4f-4b69-a1a2-dd7a8426e839 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:48.227405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d415671-62b5-44f4-b2d0-3d0de0ce172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:48.228097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d415671-62b5-44f4-b2d0-3d0de0ce172a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:48.228495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d415671-62b5-44f4-b2d0-3d0de0ce172a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:48.228775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d415671-62b5-44f4-b2d0-3d0de0ce172a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:48.243813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d415671-62b5-44f4-b2d0-3d0de0ce172a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0d415671-62b5-44f4-b2d0-3d0de0ce172a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:48.249803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:48.250685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:48.251547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 219/406] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:48.257422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d415671-62b5-44f4-b2d0-3d0de0ce172a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:48.290923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6370c08-f785-46be-a7c0-51f17ad3c07f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:48.331139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0ba4a-8983-44ba-be17-fb3b5ef3df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:48.332188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0ba4a-8983-44ba-be17-fb3b5ef3df3b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:48.332698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0ba4a-8983-44ba-be17-fb3b5ef3df3b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:48.333141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0ba4a-8983-44ba-be17-fb3b5ef3df3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:48.345981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0ba4a-8983-44ba-be17-fb3b5ef3df3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-63a0ba4a-8983-44ba-be17-fb3b5ef3df3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:48.352253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:48.353161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:48.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:48.356778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 188/407] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:48.362902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0ba4a-8983-44ba-be17-fb3b5ef3df3b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:48.400355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:48.607493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d415671-62b5-44f4-b2d0-3d0de0ce172a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:48.649200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:48.684674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0ba4a-8983-44ba-be17-fb3b5ef3df3b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:48.727225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:48.733322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:48.733322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:48.733322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:48.733322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:48.733322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:48.733322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:48.754137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 220/408] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 497 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:48.759801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:48.797318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:48.797318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:48.797318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:48.797318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:48.797740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:48.797740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:48.801321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:48.831291 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 189/409] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:48.833214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:48.941367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:48.964428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d835-4084-43ee-9e54-5608f23f9076 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:48.995962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e0f7-b204-4720-9167-95c5b6e410ab None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:49.010384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85e8c0-a14d-42db-afa4-d4edf0f40836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.011228 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85e8c0-a14d-42db-afa4-d4edf0f40836 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.011721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85e8c0-a14d-42db-afa4-d4edf0f40836 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.012064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85e8c0-a14d-42db-afa4-d4edf0f40836 None admin] PATH_INFO: `/v3/groups/81d352431a674a24aee48ba4125db057` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.013549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85e8c0-a14d-42db-afa4-d4edf0f40836 None admin] RBAC: Authorizing `identity:get_group(group_id=81d352431a674a24aee48ba4125db057)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:49.015396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85e8c0-a14d-42db-afa4-d4edf0f40836 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:49.022694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 190/410] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:48 2022] GET /identity/v3/groups/81d352431a674a24aee48ba4125db057 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:49.024400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd93c0-d8f9-4509-8bfd-9ad597f060c4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.025716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd93c0-d8f9-4509-8bfd-9ad597f060c4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.026210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd93c0-d8f9-4509-8bfd-9ad597f060c4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.033248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 191/411] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:49.035361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd93c0-d8f9-4509-8bfd-9ad597f060c4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:49.043878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b2828-498f-4ebe-978f-7b882bb39e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.044772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2828-498f-4ebe-978f-7b882bb39e59 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.045233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2828-498f-4ebe-978f-7b882bb39e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.045734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2828-498f-4ebe-978f-7b882bb39e59 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.047569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b2828-498f-4ebe-978f-7b882bb39e59 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:49.050850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b2828-498f-4ebe-978f-7b882bb39e59 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:49.053550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 221/412] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:48 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:49.055720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b00ce-6180-427b-9b24-2bbf29dd5383 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.056194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b00ce-6180-427b-9b24-2bbf29dd5383 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.056648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b00ce-6180-427b-9b24-2bbf29dd5383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.080382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1e9b00ce-6180-427b-9b24-2bbf29dd5383 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:49.081976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1e9b00ce-6180-427b-9b24-2bbf29dd5383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:49.120127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:49.156191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 222/413] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:49.162147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b00ce-6180-427b-9b24-2bbf29dd5383 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:49.178022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd93c0-d8f9-4509-8bfd-9ad597f060c4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:49.222990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de83d38-571a-441e-bbd6-fe50e9369b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.224381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de83d38-571a-441e-bbd6-fe50e9369b7a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.225336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de83d38-571a-441e-bbd6-fe50e9369b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.226128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de83d38-571a-441e-bbd6-fe50e9369b7a None admin] PATH_INFO: `/v3/projects/25016d71f554431cbc0d0222ca313083` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.227765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de83d38-571a-441e-bbd6-fe50e9369b7a None admin] RBAC: Authorizing `identity:get_project(project_id=25016d71f554431cbc0d0222ca313083)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:49.230289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de83d38-571a-441e-bbd6-fe50e9369b7a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:49.232845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 192/414] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:49 2022] GET /identity/v3/projects/25016d71f554431cbc0d0222ca313083 => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:49.233848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de83d38-571a-441e-bbd6-fe50e9369b7a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:49.265929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:49.292073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b00ce-6180-427b-9b24-2bbf29dd5383 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:49.330783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4d0f7-6328-4102-872d-a3eb302e8a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.331568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4d0f7-6328-4102-872d-a3eb302e8a08 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.331849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4d0f7-6328-4102-872d-a3eb302e8a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.332124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4d0f7-6328-4102-872d-a3eb302e8a08 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.333317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4d0f7-6328-4102-872d-a3eb302e8a08 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:49.334786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4d0f7-6328-4102-872d-a3eb302e8a08 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:49.336781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 223/415] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:49 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:49.337640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4d0f7-6328-4102-872d-a3eb302e8a08 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:49.379275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de83d38-571a-441e-bbd6-fe50e9369b7a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:49.383776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:49.429742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226236f-095d-4399-89d8-ef9c1015fec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.430865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226236f-095d-4399-89d8-ef9c1015fec9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.431379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226236f-095d-4399-89d8-ef9c1015fec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.432121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226236f-095d-4399-89d8-ef9c1015fec9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.447026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226236f-095d-4399-89d8-ef9c1015fec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:49.451887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4d0f7-6328-4102-872d-a3eb302e8a08 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-0226236f-095d-4399-89d8-ef9c1015fec9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:49.457515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:49.458441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:49.459250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:49.463555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 193/416] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:49.464394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226236f-095d-4399-89d8-ef9c1015fec9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:49.498534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e37fc5-68d6-450a-af48-c62b050a6660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.499668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e37fc5-68d6-450a-af48-c62b050a6660 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.500182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e37fc5-68d6-450a-af48-c62b050a6660 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.500718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e37fc5-68d6-450a-af48-c62b050a6660 None admin] PATH_INFO: `/v3/projects/25016d71f554431cbc0d0222ca313083/groups/81d352431a674a24aee48ba4125db057/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.503233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e37fc5-68d6-450a-af48-c62b050a6660 None admin] RBAC: Authorizing `identity:create_grant(project_id=25016d71f554431cbc0d0222ca313083, group_id=81d352431a674a24aee48ba4125db057, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:49.505017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e37fc5-68d6-450a-af48-c62b050a6660 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:49.532370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 224/417] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jul 28 06:23:49 2022] PUT /identity/v3/projects/25016d71f554431cbc0d0222ca313083/groups/81d352431a674a24aee48ba4125db057/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:49.535121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e37fc5-68d6-450a-af48-c62b050a6660 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:49.607527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:49.659949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226236f-095d-4399-89d8-ef9c1015fec9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:49.680403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:49.685566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e37fc5-68d6-450a-af48-c62b050a6660 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:49.728838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.730039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.730506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.731344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.732260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:49.732752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:49.734924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:49.747482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 225/418] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:49.757446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:49.762549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f878183-f0da-4ccb-8b4a-e60198ee5090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.763791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f878183-f0da-4ccb-8b4a-e60198ee5090 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.764219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f878183-f0da-4ccb-8b4a-e60198ee5090 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.764636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f878183-f0da-4ccb-8b4a-e60198ee5090 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/groups/b6169fd52d40419b87370474c1e963b0/roles/74f371799fd245bf987e798f5c342e6a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.768226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f878183-f0da-4ccb-8b4a-e60198ee5090 None admin] RBAC: Authorizing `identity:create_grant(project_id=0fe9b2593d1947c19cb976bc9c6e51f1, group_id=b6169fd52d40419b87370474c1e963b0, role_id=74f371799fd245bf987e798f5c342e6a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:49.775929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f878183-f0da-4ccb-8b4a-e60198ee5090 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:49.798475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 194/419] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Jul 28 06:23:49 2022] PUT /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1/groups/b6169fd52d40419b87370474c1e963b0/roles/74f371799fd245bf987e798f5c342e6a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:23:49.896747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2564-5eaf-408e-b1ec-6fa2240fd881 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:49.940831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0db6ee-71d3-4ee4-b13e-100f806a8676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:49.941493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0db6ee-71d3-4ee4-b13e-100f806a8676 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:49.941831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0db6ee-71d3-4ee4-b13e-100f806a8676 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:49.942096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0db6ee-71d3-4ee4-b13e-100f806a8676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:49.982078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:50.024560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0db6ee-71d3-4ee4-b13e-100f806a8676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:50.026399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0db6ee-71d3-4ee4-b13e-100f806a8676 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:50.030437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e103f6-a97e-4183-a2a9-e4b18b3cd01d None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.030730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e103f6-a97e-4183-a2a9-e4b18b3cd01d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.031011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e103f6-a97e-4183-a2a9-e4b18b3cd01d None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.032389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 195/420] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:50.038717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf74d23-b8b1-4cd6-80ec-7976453fa9e4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.038900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf74d23-b8b1-4cd6-80ec-7976453fa9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.039067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf74d23-b8b1-4cd6-80ec-7976453fa9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.070543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2bf74d23-b8b1-4cd6-80ec-7976453fa9e4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:50.073514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2bf74d23-b8b1-4cd6-80ec-7976453fa9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:50.166720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 226/421] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:23:50.170529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 196/422] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:50.178503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0db6ee-71d3-4ee4-b13e-100f806a8676 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:50.269565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf74d23-b8b1-4cd6-80ec-7976453fa9e4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:50.280558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:50.364097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0db6ee-71d3-4ee4-b13e-100f806a8676 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:50.409689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:50.445017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf74d23-b8b1-4cd6-80ec-7976453fa9e4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:50.447485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd85e80-0bab-422b-ae9c-e0aeb5555b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:50.449195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd85e80-0bab-422b-ae9c-e0aeb5555b10 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.449519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd85e80-0bab-422b-ae9c-e0aeb5555b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.452292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd85e80-0bab-422b-ae9c-e0aeb5555b10 None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.452292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd85e80-0bab-422b-ae9c-e0aeb5555b10 None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:50.454535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd85e80-0bab-422b-ae9c-e0aeb5555b10 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:50.456117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 227/423] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:50 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:50.462070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad05cc5-f0c3-4678-99f2-fe7f39c0a6a0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.462353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad05cc5-f0c3-4678-99f2-fe7f39c0a6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.462631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad05cc5-f0c3-4678-99f2-fe7f39c0a6a0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.464295 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 228/424] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:50.466123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad05cc5-f0c3-4678-99f2-fe7f39c0a6a0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:50.495007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab469872-4625-425a-92cd-03fc37ad116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:50.496396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab469872-4625-425a-92cd-03fc37ad116f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.497144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab469872-4625-425a-92cd-03fc37ad116f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.497624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab469872-4625-425a-92cd-03fc37ad116f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.510717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab469872-4625-425a-92cd-03fc37ad116f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ab469872-4625-425a-92cd-03fc37ad116f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:50.517702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:50.518796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:50.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:50.522491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 197/425] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:50.524732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a711ac0-620e-4c2d-89e8-a77d4c0c4538 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.525321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a711ac0-620e-4c2d-89e8-a77d4c0c4538 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.525745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a711ac0-620e-4c2d-89e8-a77d4c0c4538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.547814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6a711ac0-620e-4c2d-89e8-a77d4c0c4538 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:50.550034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6a711ac0-620e-4c2d-89e8-a77d4c0c4538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:50.593945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad05cc5-f0c3-4678-99f2-fe7f39c0a6a0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:50.631051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 198/426] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:50.632173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a711ac0-620e-4c2d-89e8-a77d4c0c4538 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:50.638586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:50.646624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f9734-eecd-4693-8eda-e591b7f39542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:50.647525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f9734-eecd-4693-8eda-e591b7f39542 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.648062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f9734-eecd-4693-8eda-e591b7f39542 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.648484 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f9734-eecd-4693-8eda-e591b7f39542 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.649881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f9734-eecd-4693-8eda-e591b7f39542 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:50.651606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f9734-eecd-4693-8eda-e591b7f39542 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:50.653766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 229/427] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:50 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:50.658306 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f9734-eecd-4693-8eda-e591b7f39542 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:50.706512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:50.753146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a711ac0-620e-4c2d-89e8-a77d4c0c4538 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:50.768852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f9734-eecd-4693-8eda-e591b7f39542 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:50.791188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:50.791949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.792301 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.792644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.793468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:50.793841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:50.795479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:50.805519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 199/428] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:50.806710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:50.807853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d1a15-0874-47bd-b5f3-177013edd79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:50.808710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d1a15-0874-47bd-b5f3-177013edd79c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.809170 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d1a15-0874-47bd-b5f3-177013edd79c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.809773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d1a15-0874-47bd-b5f3-177013edd79c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.822123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d1a15-0874-47bd-b5f3-177013edd79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-639d1a15-0874-47bd-b5f3-177013edd79c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:50.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:50.829410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 06:23:50.830243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:50.833574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 230/429] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:50.834652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d1a15-0874-47bd-b5f3-177013edd79c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:50.897435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:50.918298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af2dc1-c252-497d-80bc-3b1170c91462 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:50.942988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:50.945736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d1a15-0874-47bd-b5f3-177013edd79c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:50.954873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:50.956016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.956330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.956667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:50.957610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] RBAC: Adding query filter params (group.id=b6169fd52d40419b87370474c1e963b0, role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:50.957875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:50.959912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:50.973482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 200/430] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:50 2022] GET /identity/v3/role_assignments?group.id=b6169fd52d40419b87370474c1e963b0&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=551979dba2834cf397f2c386c2c64b2d => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:50.974722 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:50.986478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a3bdf-daef-418f-a336-36804143a9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:50.987967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a3bdf-daef-418f-a336-36804143a9dc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:50.988402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a3bdf-daef-418f-a336-36804143a9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:50.988864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a3bdf-daef-418f-a336-36804143a9dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:51.087838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a3bdf-daef-418f-a336-36804143a9dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:51.089992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a3bdf-daef-418f-a336-36804143a9dc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:51.101704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf171dc-e193-4e81-8351-202abbd14c57 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:51.140832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:51.141620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:51.141836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:51.142273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:51.143059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] RBAC: Adding query filter params (name=member) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:51.143394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:51.145101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:51.154499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 201/431] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:23:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:51.159931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:51.162116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:51.210574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 231/432] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:51.218467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:51.219139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a3bdf-daef-418f-a336-36804143a9dc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:51.281412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29399773-5e13-4726-9667-6fad45e2a3fb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:51.312650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92b512-b5bb-4c6a-b5e1-148801560e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:51.313401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92b512-b5bb-4c6a-b5e1-148801560e83 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:51.313583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92b512-b5bb-4c6a-b5e1-148801560e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:51.313866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92b512-b5bb-4c6a-b5e1-148801560e83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:51.361186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a3bdf-daef-418f-a336-36804143a9dc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:51.389424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92b512-b5bb-4c6a-b5e1-148801560e83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:51.391104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92b512-b5bb-4c6a-b5e1-148801560e83 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:51.399754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5598057-3267-45b0-86fa-953b1a7a2eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:51.400748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5598057-3267-45b0-86fa-953b1a7a2eae None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:51.401256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5598057-3267-45b0-86fa-953b1a7a2eae None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:51.401761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5598057-3267-45b0-86fa-953b1a7a2eae None admin] PATH_INFO: `/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:51.403209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5598057-3267-45b0-86fa-953b1a7a2eae None admin] RBAC: Authorizing `identity:get_project(project_id=89d1fa86df9a40fe96f2a7554edbbaed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:51.404952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5598057-3267-45b0-86fa-953b1a7a2eae None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:51.407351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 232/433] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:51 2022] GET /identity/v3/projects/89d1fa86df9a40fe96f2a7554edbbaed => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:51.412993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5598057-3267-45b0-86fa-953b1a7a2eae None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:51.451470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:51.454044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:51.517104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 202/434] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:23:51.519225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92b512-b5bb-4c6a-b5e1-148801560e83 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:51.546885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5598057-3267-45b0-86fa-953b1a7a2eae None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:51.585866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8942ebe-d70c-4752-999f-29cfe8394799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:51.587297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8942ebe-d70c-4752-999f-29cfe8394799 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:51.588163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8942ebe-d70c-4752-999f-29cfe8394799 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:51.588931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8942ebe-d70c-4752-999f-29cfe8394799 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:51.591708 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8942ebe-d70c-4752-999f-29cfe8394799 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:51.594883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8942ebe-d70c-4752-999f-29cfe8394799 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:51.600990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 233/435] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:51 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:51.610436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8942ebe-d70c-4752-999f-29cfe8394799 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:51.630727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92b512-b5bb-4c6a-b5e1-148801560e83 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:51.669137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b360d18-e43b-468b-a978-5cd43946bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:51.669858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b360d18-e43b-468b-a978-5cd43946bbae None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:51.670143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b360d18-e43b-468b-a978-5cd43946bbae None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:51.670443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b360d18-e43b-468b-a978-5cd43946bbae None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:51.671821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b360d18-e43b-468b-a978-5cd43946bbae None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:51.673277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b360d18-e43b-468b-a978-5cd43946bbae None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:51.680334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 203/436] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:51 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:51.681795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b360d18-e43b-468b-a978-5cd43946bbae None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:51.702610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:51.722888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:51.731391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8942ebe-d70c-4752-999f-29cfe8394799 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:51.770918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:51.772428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:51.772917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:51.773457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:51.774450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] RBAC: Adding query filter params (group.id=b6169fd52d40419b87370474c1e963b0, role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:51.775082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:51.776834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:51.784120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 234/437] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:51 2022] GET /identity/v3/role_assignments?group.id=b6169fd52d40419b87370474c1e963b0&scope.project.id=89d1fa86df9a40fe96f2a7554edbbaed&role.id=74f371799fd245bf987e798f5c342e6a => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:51.796704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b360d18-e43b-468b-a978-5cd43946bbae None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:51.839129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d933b67-951c-4482-8156-e924a6840d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:51.839854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b67-951c-4482-8156-e924a6840d06 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:51.840195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b67-951c-4482-8156-e924a6840d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:51.840466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b67-951c-4482-8156-e924a6840d06 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:51.841693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d933b67-951c-4482-8156-e924a6840d06 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:51.843174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d933b67-951c-4482-8156-e924a6840d06 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:51.849516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 204/438] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:51 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:51.852209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:51.975967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997082e-bc34-43dc-92f6-f3ba2709f3f2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:52.011654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087d22a-2db0-4cf7-97f3-c5fefbe82c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:52.012280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087d22a-2db0-4cf7-97f3-c5fefbe82c2b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.012562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087d22a-2db0-4cf7-97f3-c5fefbe82c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.012884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087d22a-2db0-4cf7-97f3-c5fefbe82c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.013706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087d22a-2db0-4cf7-97f3-c5fefbe82c2b None admin] RBAC: Adding query filter params (group.id=b6169fd52d40419b87370474c1e963b0, role.id=551979dba2834cf397f2c386c2c64b2d, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:52.014015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087d22a-2db0-4cf7-97f3-c5fefbe82c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:52.015437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087d22a-2db0-4cf7-97f3-c5fefbe82c2b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:52.023785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 235/439] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:51 2022] GET /identity/v3/role_assignments?group.id=b6169fd52d40419b87370474c1e963b0&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=551979dba2834cf397f2c386c2c64b2d => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:52.109444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9fd7e-3614-4ad6-9119-c88b7dee490b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.109769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9fd7e-3614-4ad6-9119-c88b7dee490b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.109960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9fd7e-3614-4ad6-9119-c88b7dee490b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.111661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 205/440] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:52.119606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a86e7-aac2-49b9-a9b5-0ec6e26c2290 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.119606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a86e7-aac2-49b9-a9b5-0ec6e26c2290 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.119606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a86e7-aac2-49b9-a9b5-0ec6e26c2290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.140839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f86a86e7-aac2-49b9-a9b5-0ec6e26c2290 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:52.143436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d68b-69fa-471f-b840-ca329e62832d None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.143603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f86a86e7-aac2-49b9-a9b5-0ec6e26c2290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:52.143684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d68b-69fa-471f-b840-ca329e62832d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.143684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d68b-69fa-471f-b840-ca329e62832d None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.144383 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 206/441] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:52.149981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abdff2-a439-4dfd-b27b-3f120d244f79 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.150319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abdff2-a439-4dfd-b27b-3f120d244f79 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.150538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abdff2-a439-4dfd-b27b-3f120d244f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.166400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:52.175588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-76abdff2-a439-4dfd-b27b-3f120d244f79 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:52.176943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-76abdff2-a439-4dfd-b27b-3f120d244f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:52.240687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 236/442] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:52.260134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 207/443] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:52.265441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a86e7-aac2-49b9-a9b5-0ec6e26c2290 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:52.284595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abdff2-a439-4dfd-b27b-3f120d244f79 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:52.301744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:52.384517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a86e7-aac2-49b9-a9b5-0ec6e26c2290 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:52.404557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abdff2-a439-4dfd-b27b-3f120d244f79 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:52.429022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae803c3-9fd1-4922-bf2c-f198ab10d1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:52.429740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae803c3-9fd1-4922-bf2c-f198ab10d1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.430023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae803c3-9fd1-4922-bf2c-f198ab10d1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.430299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae803c3-9fd1-4922-bf2c-f198ab10d1b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.442496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae803c3-9fd1-4922-bf2c-f198ab10d1b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:52.446019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48cb40-db9e-49a0-8b05-603dd61b0b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:52.446740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48cb40-db9e-49a0-8b05-603dd61b0b38 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.447026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48cb40-db9e-49a0-8b05-603dd61b0b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.447299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48cb40-db9e-49a0-8b05-603dd61b0b38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-bae803c3-9fd1-4922-bf2c-f198ab10d1b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:52.447499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:52.448456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:52.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 237/444] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:52.454861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae803c3-9fd1-4922-bf2c-f198ab10d1b7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:52.466765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48cb40-db9e-49a0-8b05-603dd61b0b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-7f48cb40-db9e-49a0-8b05-603dd61b0b38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:52.474375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:52.475289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:23:52.476117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:52.476735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:52.476806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 208/445] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:52.481545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48cb40-db9e-49a0-8b05-603dd61b0b38 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:52.559295 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:52.566651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae803c3-9fd1-4922-bf2c-f198ab10d1b7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:52.585163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48cb40-db9e-49a0-8b05-603dd61b0b38 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:52.602588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:52.603203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.603482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.603755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.604469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:52.604776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:52.606171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:52.614351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 238/446] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:52.619112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:52.619728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.619833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:52.620011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.620282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.620998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:52.621315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:52.622686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:52.631718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 209/447] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:52.636798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:52.694903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:52.720628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6af82-c936-4487-aeda-8e8fbb74cfd6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:52.739994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf02b25-9ea4-4ec9-83e6-be837d8b3e61 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:52.754683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10126a2-b0e7-451a-af0f-7198e922d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:52.755301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10126a2-b0e7-451a-af0f-7198e922d5bd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.755584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10126a2-b0e7-451a-af0f-7198e922d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.755859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10126a2-b0e7-451a-af0f-7198e922d5bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.774594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6244246-536e-48bb-8dfc-9ddee96ebaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:52.775209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6244246-536e-48bb-8dfc-9ddee96ebaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:52.775491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6244246-536e-48bb-8dfc-9ddee96ebaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:52.775766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6244246-536e-48bb-8dfc-9ddee96ebaf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:52.779108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:52.825498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10126a2-b0e7-451a-af0f-7198e922d5bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:52.826866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10126a2-b0e7-451a-af0f-7198e922d5bd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:52.846153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6244246-536e-48bb-8dfc-9ddee96ebaf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:52.848062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6244246-536e-48bb-8dfc-9ddee96ebaf2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:52.899970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:52.926208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 239/448] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:23:52.931809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10126a2-b0e7-451a-af0f-7198e922d5bd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:52.947288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 210/449] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 06:23:52.953323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6244246-536e-48bb-8dfc-9ddee96ebaf2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:52.985772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:53.031660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10126a2-b0e7-451a-af0f-7198e922d5bd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:53.056100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6244246-536e-48bb-8dfc-9ddee96ebaf2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:53.065560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0993d84-5ae6-423e-901e-22c879606a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:53.066213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0993d84-5ae6-423e-901e-22c879606a33 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:53.066490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0993d84-5ae6-423e-901e-22c879606a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:53.066757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0993d84-5ae6-423e-901e-22c879606a33 None admin] PATH_INFO: `/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:53.067907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0993d84-5ae6-423e-901e-22c879606a33 None admin] RBAC: Authorizing `identity:get_project(project_id=0fe9b2593d1947c19cb976bc9c6e51f1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:53.069365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0993d84-5ae6-423e-901e-22c879606a33 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:53.071005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 240/450] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:52 2022] GET /identity/v3/projects/0fe9b2593d1947c19cb976bc9c6e51f1 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:53.076670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0993d84-5ae6-423e-901e-22c879606a33 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:53.090253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2258e-0ece-4b57-a36e-ad103773614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:53.090887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2258e-0ece-4b57-a36e-ad103773614d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:53.091170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2258e-0ece-4b57-a36e-ad103773614d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:53.091442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2258e-0ece-4b57-a36e-ad103773614d None admin] PATH_INFO: `/v3/projects/25016d71f554431cbc0d0222ca313083` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:53.092583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2258e-0ece-4b57-a36e-ad103773614d None admin] RBAC: Authorizing `identity:get_project(project_id=25016d71f554431cbc0d0222ca313083)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:53.094020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2258e-0ece-4b57-a36e-ad103773614d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:53.095659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 211/451] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:23:52 2022] GET /identity/v3/projects/25016d71f554431cbc0d0222ca313083 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:53.101723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2258e-0ece-4b57-a36e-ad103773614d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:53.119258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:53.176883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0993d84-5ae6-423e-901e-22c879606a33 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:53.203665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2258e-0ece-4b57-a36e-ad103773614d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:53.206965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:53.210961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd63d9f-01d2-4ac5-8eed-70a700f58357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:53.211598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd63d9f-01d2-4ac5-8eed-70a700f58357 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:53.211881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd63d9f-01d2-4ac5-8eed-70a700f58357 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:53.212148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd63d9f-01d2-4ac5-8eed-70a700f58357 None admin] PATH_INFO: `/v3/groups/b6169fd52d40419b87370474c1e963b0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:53.213308 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd63d9f-01d2-4ac5-8eed-70a700f58357 None admin] RBAC: Authorizing `identity:get_group(group_id=b6169fd52d40419b87370474c1e963b0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:53.214797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd63d9f-01d2-4ac5-8eed-70a700f58357 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:53.216362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 241/452] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:53 2022] GET /identity/v3/groups/b6169fd52d40419b87370474c1e963b0 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:53.222682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd63d9f-01d2-4ac5-8eed-70a700f58357 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:53.240719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f11fef-ea9a-4eb1-a14e-99ce9691c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:53.241392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f11fef-ea9a-4eb1-a14e-99ce9691c682 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:53.241677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f11fef-ea9a-4eb1-a14e-99ce9691c682 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:53.241953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f11fef-ea9a-4eb1-a14e-99ce9691c682 None admin] PATH_INFO: `/v3/groups/81d352431a674a24aee48ba4125db057` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:53.243115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f11fef-ea9a-4eb1-a14e-99ce9691c682 None admin] RBAC: Authorizing `identity:get_group(group_id=81d352431a674a24aee48ba4125db057)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:53.244549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f11fef-ea9a-4eb1-a14e-99ce9691c682 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:53.246186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 212/453] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Jul 28 06:23:53 2022] GET /identity/v3/groups/81d352431a674a24aee48ba4125db057 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:53.251901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f11fef-ea9a-4eb1-a14e-99ce9691c682 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:53.319571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd63d9f-01d2-4ac5-8eed-70a700f58357 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:53.332116 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:53.354939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f11fef-ea9a-4eb1-a14e-99ce9691c682 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:53.356865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40181cc-0125-4739-aca8-f9e8083091d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:53.357515 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40181cc-0125-4739-aca8-f9e8083091d2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:53.357804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40181cc-0125-4739-aca8-f9e8083091d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:53.358075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40181cc-0125-4739-aca8-f9e8083091d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:53.358860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40181cc-0125-4739-aca8-f9e8083091d2 None admin] RBAC: Adding query filter params (group.id=b6169fd52d40419b87370474c1e963b0, role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:53.359173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40181cc-0125-4739-aca8-f9e8083091d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:53.360585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40181cc-0125-4739-aca8-f9e8083091d2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:53.366521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 242/454] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:53 2022] GET /identity/v3/role_assignments?group.id=b6169fd52d40419b87370474c1e963b0&scope.project.id=0fe9b2593d1947c19cb976bc9c6e51f1&role.id=74f371799fd245bf987e798f5c342e6a => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:53.390576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b005c-41ba-434a-a4a5-24d8d7bfa7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:53.391217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b005c-41ba-434a-a4a5-24d8d7bfa7ab None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:53.391502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b005c-41ba-434a-a4a5-24d8d7bfa7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:53.391776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b005c-41ba-434a-a4a5-24d8d7bfa7ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:53.392590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b005c-41ba-434a-a4a5-24d8d7bfa7ab None admin] RBAC: Adding query filter params (group.id=81d352431a674a24aee48ba4125db057, role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=25016d71f554431cbc0d0222ca313083) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:53.392904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b005c-41ba-434a-a4a5-24d8d7bfa7ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:53.395163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b005c-41ba-434a-a4a5-24d8d7bfa7ab None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:53.401911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 213/455] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Jul 28 06:23:53 2022] GET /identity/v3/role_assignments?group.id=81d352431a674a24aee48ba4125db057&scope.project.id=25016d71f554431cbc0d0222ca313083&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:55.856011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe11635-5bb7-4049-b562-711046138a9c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:55.856519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe11635-5bb7-4049-b562-711046138a9c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:55.856623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe11635-5bb7-4049-b562-711046138a9c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:55.858143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 243/456] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:55.864739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ea4b-8ccc-4bc5-a541-1277b8b94376 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:55.865603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ea4b-8ccc-4bc5-a541-1277b8b94376 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:55.866077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ea4b-8ccc-4bc5-a541-1277b8b94376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:55.894723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4d46ea4b-8ccc-4bc5-a541-1277b8b94376 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:55.896349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4d46ea4b-8ccc-4bc5-a541-1277b8b94376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:55.937926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:55.995914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 214/457] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:56.032152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe11635-5bb7-4049-b562-711046138a9c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:56.086661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949de4-0b9f-4764-8f1c-98296525b4e3 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.087400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949de4-0b9f-4764-8f1c-98296525b4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.088157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949de4-0b9f-4764-8f1c-98296525b4e3 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.090414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 215/458] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:56.095172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb3ee-01c4-4e68-a491-28b0c8e678df None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.095726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb3ee-01c4-4e68-a491-28b0c8e678df None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.096873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb3ee-01c4-4e68-a491-28b0c8e678df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.124443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-835fb3ee-01c4-4e68-a491-28b0c8e678df None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:56.127152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-835fb3ee-01c4-4e68-a491-28b0c8e678df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:56.213193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe11635-5bb7-4049-b562-711046138a9c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:56.232357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 216/459] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:56.267624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fb3ee-01c4-4e68-a491-28b0c8e678df None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:56.269100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:56.269719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.270002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.270279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.285382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:56.286991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:23:56.313313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:56.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:56.315186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:23:56.315186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:23:56.315186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:23:56.315186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:56.315186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 244/460] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:56.320437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:56.367005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:56.387355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:56.394832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fb3ee-01c4-4e68-a491-28b0c8e678df None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:56.441476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:56.442517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.444103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.444576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.470188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:56.472534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:56.494328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09333f-b360-4164-8ebd-99f71918a90c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:23:56.503873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:56.504757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:56.505654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:23:56.505654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:23:56.505654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:23:56.505654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:56.507542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 217/461] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:56.514608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:56.539860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf4aca-17b0-4267-a470-2b2808a7c183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:56.540483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf4aca-17b0-4267-a470-2b2808a7c183 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.540763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf4aca-17b0-4267-a470-2b2808a7c183 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.541036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf4aca-17b0-4267-a470-2b2808a7c183 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.541765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf4aca-17b0-4267-a470-2b2808a7c183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:56.542069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf4aca-17b0-4267-a470-2b2808a7c183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:56.543851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf4aca-17b0-4267-a470-2b2808a7c183 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:56.560212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 245/462] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:56.566331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8afee1-469b-4138-9d16-38e39c0bfe65 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.566331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8afee1-469b-4138-9d16-38e39c0bfe65 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.566331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8afee1-469b-4138-9d16-38e39c0bfe65 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.567815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 246/463] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:56.568525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8afee1-469b-4138-9d16-38e39c0bfe65 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:56.635332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa3b2e-5bf5-4275-99cc-2d5ab045f78f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:56.676775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d804c-f0ca-41d4-b4aa-83f1bbe2b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:56.677324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d804c-f0ca-41d4-b4aa-83f1bbe2b335 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.677630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d804c-f0ca-41d4-b4aa-83f1bbe2b335 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.677733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:56.678256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d804c-f0ca-41d4-b4aa-83f1bbe2b335 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.678963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d804c-f0ca-41d4-b4aa-83f1bbe2b335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:56.679266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d804c-f0ca-41d4-b4aa-83f1bbe2b335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:56.680744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d804c-f0ca-41d4-b4aa-83f1bbe2b335 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:56.693899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 218/464] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:56.695107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ffcd3-1713-4f33-8aae-edb93d0b5b7d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.695382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ffcd3-1713-4f33-8aae-edb93d0b5b7d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.695651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ffcd3-1713-4f33-8aae-edb93d0b5b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.700288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8afee1-469b-4138-9d16-38e39c0bfe65 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:56.713869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-795ffcd3-1713-4f33-8aae-edb93d0b5b7d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:56.715041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-795ffcd3-1713-4f33-8aae-edb93d0b5b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:56.716998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:56.740026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd9728-7b0b-4b87-a54c-d567c02822f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:56.740646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd9728-7b0b-4b87-a54c-d567c02822f8 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.741069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd9728-7b0b-4b87-a54c-d567c02822f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.741387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd9728-7b0b-4b87-a54c-d567c02822f8 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.742218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd9728-7b0b-4b87-a54c-d567c02822f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:56.743653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd9728-7b0b-4b87-a54c-d567c02822f8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:56.769319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 247/465] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:56 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:56.769911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd9728-7b0b-4b87-a54c-d567c02822f8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:56.825613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 219/466] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:56.839522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6d2de-2617-4e21-8823-7942b794a4ae None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.839815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6d2de-2617-4e21-8823-7942b794a4ae None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.840056 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6d2de-2617-4e21-8823-7942b794a4ae None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.841789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 220/467] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:56.852281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaca35-d6ff-4f04-8d4e-b09b1140b7fd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.852902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaca35-d6ff-4f04-8d4e-b09b1140b7fd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.853459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaca35-d6ff-4f04-8d4e-b09b1140b7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.874459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5dcaca35-d6ff-4f04-8d4e-b09b1140b7fd None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:56.876759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5dcaca35-d6ff-4f04-8d4e-b09b1140b7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:56.911094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd9728-7b0b-4b87-a54c-d567c02822f8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:56.952233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f5111-679a-4d68-9f43-60e4a26460f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:56.952778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5111-679a-4d68-9f43-60e4a26460f3 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:56.953109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5111-679a-4d68-9f43-60e4a26460f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:56.953405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f5111-679a-4d68-9f43-60e4a26460f3 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:56.954095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f5111-679a-4d68-9f43-60e4a26460f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:56.955565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f5111-679a-4d68-9f43-60e4a26460f3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:56.974658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 221/468] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:23:56.975609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:56.976930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaca35-d6ff-4f04-8d4e-b09b1140b7fd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:56.981328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 248/469] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:56 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:57.006634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f5111-679a-4d68-9f43-60e4a26460f3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:57.083221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:57.123553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaca35-d6ff-4f04-8d4e-b09b1140b7fd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:57.164973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f5111-679a-4d68-9f43-60e4a26460f3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:57.165233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:57.166086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.166891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.167310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.185033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:57.186541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:23:57.207158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:57.208033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:57.208899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:23:57.208899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:23:57.208899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:23:57.208899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:57.209114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:57.209114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.209263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.209893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.210435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 222/470] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:57.217500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:57.226597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:57.228468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:23:57.246742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:57.247642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:57.248496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:23:57.248496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:23:57.248496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:23:57.248496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:57.248496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 249/471] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:57.255240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:57.341453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:57.353988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d262c7-e606-4d11-ac4f-0846a7b179d8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:57.380853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:57.402242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:57.402988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.403790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.404907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:57.405214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:57.407097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:57.414836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bf43d-9aa3-490f-ac41-40d6bc10ef2c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:57.424270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 223/472] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:57.429525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:57.457382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c457f-d0a1-4efc-9570-9071307d3f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:57.458354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c457f-d0a1-4efc-9570-9071307d3f82 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.458911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c457f-d0a1-4efc-9570-9071307d3f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.459187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c457f-d0a1-4efc-9570-9071307d3f82 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.459893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c457f-d0a1-4efc-9570-9071307d3f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:57.460402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c457f-d0a1-4efc-9570-9071307d3f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:57.462109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c457f-d0a1-4efc-9570-9071307d3f82 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:57.475835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 250/473] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:23:57.477080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571154f-5137-4e2a-94f0-2fe5df5361a4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.481482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571154f-5137-4e2a-94f0-2fe5df5361a4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.481801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571154f-5137-4e2a-94f0-2fe5df5361a4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.483322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 251/474] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:57.484711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571154f-5137-4e2a-94f0-2fe5df5361a4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:57.551810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8715e3-d8af-4c5f-b4ef-faba04cf21d4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:57.593559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571154f-5137-4e2a-94f0-2fe5df5361a4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:57.597861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c3239-ca44-4afd-9799-02a36dfc99d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:57.598702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c3239-ca44-4afd-9799-02a36dfc99d8 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.599131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c3239-ca44-4afd-9799-02a36dfc99d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.599585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c3239-ca44-4afd-9799-02a36dfc99d8 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.600445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c3239-ca44-4afd-9799-02a36dfc99d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:57.602141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c3239-ca44-4afd-9799-02a36dfc99d8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:57.617838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:57.630253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 224/475] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:57 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:23:57.631932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663edcc-b788-4bfe-a951-c2f5c792df17 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.632432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663edcc-b788-4bfe-a951-c2f5c792df17 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.632889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663edcc-b788-4bfe-a951-c2f5c792df17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.640488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa136a-7146-4f58-87eb-fd7ce2c3ba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:57.641127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa136a-7146-4f58-87eb-fd7ce2c3ba81 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.641433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa136a-7146-4f58-87eb-fd7ce2c3ba81 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.642657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa136a-7146-4f58-87eb-fd7ce2c3ba81 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.642657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa136a-7146-4f58-87eb-fd7ce2c3ba81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:57.644791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa136a-7146-4f58-87eb-fd7ce2c3ba81 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:57.656129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1663edcc-b788-4bfe-a951-c2f5c792df17 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:57.659323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1663edcc-b788-4bfe-a951-c2f5c792df17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:57.697103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:57.704593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 252/476] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:23:57 2022] POST /identity/v3/users => generated 249 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:23:57.747336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 225/477] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:57.783023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa136a-7146-4f58-87eb-fd7ce2c3ba81 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:57.916103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa136a-7146-4f58-87eb-fd7ce2c3ba81 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:57.954309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:57.954938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:57.955250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:57.955506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:57.956243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:57.957653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:57.966220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.sql.core [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 06:23:57.966220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 06:23:57.966220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [parameters: {'id': 'b789aea5cca24b229672b0efd96e0378', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 06:23:57.966220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80277) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result.read() Jul 28 06:23:57.983706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:23:57.984629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application yield resource Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise self.value Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application session.commit() Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 06:23:57.985966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.session.flush() Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application compat.raise_( Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 06:23:57.987718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:23:57.988616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result.read() Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:23:57.990483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [parameters: {'id': 'b789aea5cca24b229672b0efd96e0378', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:57.992394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 06:23:57.994160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:57.995598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 253/478] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jul 28 06:23:57 2022] POST /identity/v3/roles => generated 144 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:57.996996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663edcc-b788-4bfe-a951-c2f5c792df17 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:58.116669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663edcc-b788-4bfe-a951-c2f5c792df17 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:58.159693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728d225-865d-4873-b3f8-c9ee1a3c483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:58.160344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728d225-865d-4873-b3f8-c9ee1a3c483f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:58.160627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728d225-865d-4873-b3f8-c9ee1a3c483f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:58.160903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728d225-865d-4873-b3f8-c9ee1a3c483f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:58.171836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728d225-865d-4873-b3f8-c9ee1a3c483f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8728d225-865d-4873-b3f8-c9ee1a3c483f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:58.177072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:58.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:58.179063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 226/479] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:23:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:58.183984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:58.216098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:58.311977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef8ef6-9a83-4c26-acc4-1638ae738120 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:58.368870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457dc3-4d43-4e48-b4ca-0a5b578e86b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:58.369432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457dc3-4d43-4e48-b4ca-0a5b578e86b3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:58.369719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457dc3-4d43-4e48-b4ca-0a5b578e86b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:58.369990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457dc3-4d43-4e48-b4ca-0a5b578e86b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:58.370710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70457dc3-4d43-4e48-b4ca-0a5b578e86b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:23:58.371014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70457dc3-4d43-4e48-b4ca-0a5b578e86b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:23:58.372382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70457dc3-4d43-4e48-b4ca-0a5b578e86b3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:23:58.381570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 254/480] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:23:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:23:59.313919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ad49-c644-4035-891b-06d86d665720 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:59.314456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ad49-c644-4035-891b-06d86d665720 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:59.315385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ad49-c644-4035-891b-06d86d665720 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:59.317071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 227/481] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:23:59.322596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68e92e-eee4-45e2-8993-a903dca49a3b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:59.323028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68e92e-eee4-45e2-8993-a903dca49a3b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:59.323198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68e92e-eee4-45e2-8993-a903dca49a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:59.350309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-de68e92e-eee4-45e2-8993-a903dca49a3b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:59.351516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-de68e92e-eee4-45e2-8993-a903dca49a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:59.443323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 255/482] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:23:59.470313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3ad49-c644-4035-891b-06d86d665720 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:59.511513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:59.527245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a446fc-bd58-46e1-9250-d0e726b204c0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:59.527548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a446fc-bd58-46e1-9250-d0e726b204c0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:59.527841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a446fc-bd58-46e1-9250-d0e726b204c0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:59.529483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 256/483] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:23:59.534728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe602992-7300-46ff-92cd-6035513cbe6a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:59.535022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe602992-7300-46ff-92cd-6035513cbe6a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:59.535302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe602992-7300-46ff-92cd-6035513cbe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:59.574279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fe602992-7300-46ff-92cd-6035513cbe6a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:23:59.575951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fe602992-7300-46ff-92cd-6035513cbe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:23:59.601487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:59.647255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3ad49-c644-4035-891b-06d86d665720 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:59.675055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 257/484] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 28 06:23:59.708052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe602992-7300-46ff-92cd-6035513cbe6a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:59.720737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4276f-e44d-4c05-98f5-61849347afe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:59.722254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4276f-e44d-4c05-98f5-61849347afe1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:59.722758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4276f-e44d-4c05-98f5-61849347afe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:59.723225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4276f-e44d-4c05-98f5-61849347afe1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:59.754226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4276f-e44d-4c05-98f5-61849347afe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-04a4276f-e44d-4c05-98f5-61849347afe1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:59.759546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:59.760434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:23:59.761263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:23:59.773314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 228/485] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:23:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:23:59.774465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4276f-e44d-4c05-98f5-61849347afe1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:59.818711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe602992-7300-46ff-92cd-6035513cbe6a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:23:59.857414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbef887-9cc1-4291-a068-da28e084dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:23:59.857977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbef887-9cc1-4291-a068-da28e084dbfd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:23:59.858259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbef887-9cc1-4291-a068-da28e084dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:23:59.858532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbef887-9cc1-4291-a068-da28e084dbfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:23:59.870040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbef887-9cc1-4291-a068-da28e084dbfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3fbef887-9cc1-4291-a068-da28e084dbfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:23:59.875277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:23:59.876160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:23:59.876981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 258/486] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:23:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:23:59.886807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbef887-9cc1-4291-a068-da28e084dbfd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:23:59.918753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:59.969658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:23:59.993116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4276f-e44d-4c05-98f5-61849347afe1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.029950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.030571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.030864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.031138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.031979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:00.032291 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.034424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.042789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 229/487] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:23:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:00.048452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.118119 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbef887-9cc1-4291-a068-da28e084dbfd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.196404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e4ab2-5f55-4391-9458-9da01bc522e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.197029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4ab2-5f55-4391-9458-9da01bc522e6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.197322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4ab2-5f55-4391-9458-9da01bc522e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.197593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4ab2-5f55-4391-9458-9da01bc522e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.198311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e4ab2-5f55-4391-9458-9da01bc522e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:00.198612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e4ab2-5f55-4391-9458-9da01bc522e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.200239 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e4ab2-5f55-4391-9458-9da01bc522e6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.221554 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 259/488] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:23:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:00.223247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa75577-147e-4184-9f9c-b91ea9108c10 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.223538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa75577-147e-4184-9f9c-b91ea9108c10 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.223812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa75577-147e-4184-9f9c-b91ea9108c10 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.225315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 260/489] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:00.226857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa75577-147e-4184-9f9c-b91ea9108c10 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.236774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c2363-a8b6-405c-802f-795d3ce821d9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.278560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617df5c2-4d34-4dc6-b9f9-29adf0005cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.279256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df5c2-4d34-4dc6-b9f9-29adf0005cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.279544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df5c2-4d34-4dc6-b9f9-29adf0005cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.279906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df5c2-4d34-4dc6-b9f9-29adf0005cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.292642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617df5c2-4d34-4dc6-b9f9-29adf0005cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.294212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617df5c2-4d34-4dc6-b9f9-29adf0005cc0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.302276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-617df5c2-4d34-4dc6-b9f9-29adf0005cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:00.309475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:00.310802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:00.312896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:00.312896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:00.312896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:00.312896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:00.317040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:00.317519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 230/490] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:00.318429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ec675-25e3-400b-a430-5c63aaba2ea2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.318714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ec675-25e3-400b-a430-5c63aaba2ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.318988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ec675-25e3-400b-a430-5c63aaba2ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.338730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6e5ec675-25e3-400b-a430-5c63aaba2ea2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:00.339993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6e5ec675-25e3-400b-a430-5c63aaba2ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:00.343944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa75577-147e-4184-9f9c-b91ea9108c10 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.387332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.387895 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.388174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.388500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.399803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.401479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:00.417952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:00.419990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:00.421486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:00.421486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:00.421486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:00.421486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:00.429316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 261/491] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:00.429316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.429531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 231/492] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:00.429531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ec675-25e3-400b-a430-5c63aaba2ea2 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.548583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177a2e-484e-4d2b-9364-f7082547da97 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.552250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ec675-25e3-400b-a430-5c63aaba2ea2 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.583019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:00.583206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:00.589592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.589592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.589592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.589876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.593192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.593341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:00.593341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.593497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.593497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.593497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.593667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.593741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:00.593807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.595468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.608390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 262/493] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:00.609352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.613537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 232/494] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:00.615047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.723717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cbf9d-86aa-41e1-bbdc-f5b3fe37d8bd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.727369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddc78d-faf4-4a54-a547-508fce384af5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.758850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801a306-b954-4275-a850-adccc4d776cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.759495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801a306-b954-4275-a850-adccc4d776cb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.759775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801a306-b954-4275-a850-adccc4d776cb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.760050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801a306-b954-4275-a850-adccc4d776cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.764799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c53269-a48c-4de5-8b40-0bb7d7d8c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.765484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c53269-a48c-4de5-8b40-0bb7d7d8c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.765770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c53269-a48c-4de5-8b40-0bb7d7d8c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.766043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c53269-a48c-4de5-8b40-0bb7d7d8c0b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.770780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801a306-b954-4275-a850-adccc4d776cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:00.773895 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c53269-a48c-4de5-8b40-0bb7d7d8c0b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.775340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c53269-a48c-4de5-8b40-0bb7d7d8c0b8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6801a306-b954-4275-a850-adccc4d776cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:00.775578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:00.776435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:00.777277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:00.781720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 263/495] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:00.782713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801a306-b954-4275-a850-adccc4d776cb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-55c53269-a48c-4de5-8b40-0bb7d7d8c0b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:00.791563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:00.792529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:00.793469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:00.797412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 233/496] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:00.798632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20135989-f999-4a66-930c-93e9b27e7dbd None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.799446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20135989-f999-4a66-930c-93e9b27e7dbd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.799728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20135989-f999-4a66-930c-93e9b27e7dbd None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.801157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 234/497] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:00.805521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20135989-f999-4a66-930c-93e9b27e7dbd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.834602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:00.862215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:00.898171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801a306-b954-4275-a850-adccc4d776cb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.920636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20135989-f999-4a66-930c-93e9b27e7dbd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:00.935989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.936840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.937126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.937427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.945051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.946587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.956353 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bc127-96d2-4a01-96f3-d8c19ce71ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:00.956971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc127-96d2-4a01-96f3-d8c19ce71ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.957253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc127-96d2-4a01-96f3-d8c19ce71ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.957557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc127-96d2-4a01-96f3-d8c19ce71ed5 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.958264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc127-96d2-4a01-96f3-d8c19ce71ed5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:00.958592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc127-96d2-4a01-96f3-d8c19ce71ed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:00.959675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:00.960537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:24:00.961507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:00.962026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc127-96d2-4a01-96f3-d8c19ce71ed5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:00.962094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 264/498] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:00.962094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:00.977232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 235/499] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:00.978787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476fe33-bda8-482e-9003-2a61440efb59 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:00.979080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476fe33-bda8-482e-9003-2a61440efb59 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:00.979354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476fe33-bda8-482e-9003-2a61440efb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:00.997922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f476fe33-bda8-482e-9003-2a61440efb59 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:00.999121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f476fe33-bda8-482e-9003-2a61440efb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:01.069041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 236/500] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:01.078910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476fe33-bda8-482e-9003-2a61440efb59 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.081515 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685894a-6c09-4a4c-b95a-575502766170 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.089641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:01.120053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:01.120668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.120949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.121217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.122042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:01.122348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:01.124174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:01.135033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:01.160342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 265/501] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:24:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:01.161172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.192088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476fe33-bda8-482e-9003-2a61440efb59 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.225786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:01.226394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.227093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.227093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.227764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:01.228015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:01.229451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:01.259829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a87a39-3afe-4722-9919-e6fb6deab318 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.265211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 237/502] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:01.266141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.296570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66f8a0-f5fc-40ba-864c-29175489b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:01.297456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66f8a0-f5fc-40ba-864c-29175489b85d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.297861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66f8a0-f5fc-40ba-864c-29175489b85d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.298276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66f8a0-f5fc-40ba-864c-29175489b85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.308166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66f8a0-f5fc-40ba-864c-29175489b85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:01.309671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66f8a0-f5fc-40ba-864c-29175489b85d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6f66f8a0-f5fc-40ba-864c-29175489b85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:01.324301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:01.325139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:01.326036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:01.326036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:01.326036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:01.326036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:01.326036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 266/503] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:01.326755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd8fc3-2272-407c-aca6-28d39cd967d9 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.326859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd8fc3-2272-407c-aca6-28d39cd967d9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.327135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd8fc3-2272-407c-aca6-28d39cd967d9 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.328484 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 267/504] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:01.329022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd8fc3-2272-407c-aca6-28d39cd967d9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.347579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:01.383309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:01.386428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28b87b-125e-43a9-bf7c-9914333bc63a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.423147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71289e0c-5428-4f6a-8d5a-035ba5652014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:01.423801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289e0c-5428-4f6a-8d5a-035ba5652014 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.424864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289e0c-5428-4f6a-8d5a-035ba5652014 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.425144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289e0c-5428-4f6a-8d5a-035ba5652014 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.431980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd8fc3-2272-407c-aca6-28d39cd967d9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.435443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71289e0c-5428-4f6a-8d5a-035ba5652014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-71289e0c-5428-4f6a-8d5a-035ba5652014 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:01.441089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:01.442260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 238/505] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:01.443073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71289e0c-5428-4f6a-8d5a-035ba5652014 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.474722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073b907-12b1-43d1-a0ea-58665993fbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:01.475334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073b907-12b1-43d1-a0ea-58665993fbdf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.475615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073b907-12b1-43d1-a0ea-58665993fbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.475890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073b907-12b1-43d1-a0ea-58665993fbdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.543041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71289e0c-5428-4f6a-8d5a-035ba5652014 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.543852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073b907-12b1-43d1-a0ea-58665993fbdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:01.545358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073b907-12b1-43d1-a0ea-58665993fbdf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:01.562037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:01.576996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaae87e-8bc9-4cee-ab87-51e662d9fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:01.577626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaae87e-8bc9-4cee-ab87-51e662d9fe1d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.577909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaae87e-8bc9-4cee-ab87-51e662d9fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.578180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaae87e-8bc9-4cee-ab87-51e662d9fe1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.606084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:01.645482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 268/506] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:01.646336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073b907-12b1-43d1-a0ea-58665993fbdf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.657537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaae87e-8bc9-4cee-ab87-51e662d9fe1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:01.659618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaae87e-8bc9-4cee-ab87-51e662d9fe1d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:01.747029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073b907-12b1-43d1-a0ea-58665993fbdf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.764186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:01.775245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 239/507] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:01.776489 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed3e62-e84f-45ac-8c43-b0e9d58e0c92 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.776773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed3e62-e84f-45ac-8c43-b0e9d58e0c92 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.777044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed3e62-e84f-45ac-8c43-b0e9d58e0c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.788618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:01.789685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.789975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.790254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.790949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:01.791258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:01.792612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:01.794417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-36ed3e62-e84f-45ac-8c43-b0e9d58e0c92 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:01.795537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-36ed3e62-e84f-45ac-8c43-b0e9d58e0c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:01.804252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 269/508] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:01.805047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.854764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:01.866257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 240/509] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:01.867058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed3e62-e84f-45ac-8c43-b0e9d58e0c92 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.906346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5f3bd-85c8-48d7-b6a9-bff47d55838e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.940538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:01.941153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:01.941462 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:01.941734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:01.942448 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:01.942749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:01.944117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:01.952212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 270/510] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:01.952991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:01.975373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed3e62-e84f-45ac-8c43-b0e9d58e0c92 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:01.991991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.010466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.011116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.011402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.011678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.022445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.023951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:02.040052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:02.041149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:02.042083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:02.042083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:02.042083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:02.042083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:02.042083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 241/511] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:02.042427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.058418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b68608-822a-4874-9e20-fd4ee18508d5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.085431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.093215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.093885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.094172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.094450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.105380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.106842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:02.121373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:02.122212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:02.123121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:02.123121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:02.123121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:02.123121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:02.123121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 271/512] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:02.123394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.147269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe19f95-796b-4521-af17-a71e85184fc0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.181019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.181688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.181967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.182246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.189667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.191150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:02.202464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:02.203559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 242/513] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:02.204723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.222922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.232618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875f0c4-a7f6-4705-931b-f61b9bf9ded3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.265655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.266286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.266565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.266844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.277629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.279064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:02.293606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:02.294590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:02.295651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:02.295651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:02.295651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:02.295651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:02.295651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 272/514] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:02.295651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.310519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f7d7-bbcb-47a3-b88b-369c93153646 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.311967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.345460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.346149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.346430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.346707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.356964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.358433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:02.391638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:02.392981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:02.393889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:02.393889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:02.393889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:02.393889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:02.393889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 243/515] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:02.394166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b5c32-ab82-430a-bbd4-b654f67da969 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.394511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.434958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.435694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.435993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.436342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.437038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:02.437362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.438717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.450698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 273/516] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:02.451521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.458164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.503339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9eac2-6f08-4511-a31b-24bda07fbfc6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.538069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.538789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.539133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.539486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.540273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:02.540744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.542309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.554973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 244/517] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:02.556749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485a531-033b-4a7d-b683-c0e63a82fc25 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.559903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.575213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.590965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.591577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.591857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.592125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.592901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:02.593205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.594648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.611644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 274/518] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:02.612394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.661252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dc0fc-e28c-4394-96c5-a476c2c83963 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.673451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.694828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.695447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.695729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.695997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.696679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:02.697057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.698454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.710019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 245/519] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:02.710611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.719286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd08de-2a5a-479f-8f70-09c350169413 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.758869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.759489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.759769 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.760040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.760725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:02.761025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.762430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.773917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 275/520] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:02.774739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.787984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.812897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd2cea-a908-4c2f-a947-3cf0b2a00065 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.846581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.847154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.847429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.847696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.854170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.855620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:02.868093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:02.868937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:02.869823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 246/521] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:02.870292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.878715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9411f-a4d0-4283-9c20-af24d4220a48 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:02.898604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:02.913871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:02.914282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:02.914567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:02.914843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:02.921570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:02.923008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:02.935653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:02.936586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:02.937561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 276/522] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:02.938024 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:02.975214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea779a2b-0202-4390-a1cf-62cdebce87f5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.009252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a958f-aea9-4221-b1b2-cd4f32fa240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.009885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a958f-aea9-4221-b1b2-cd4f32fa240d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.010161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a958f-aea9-4221-b1b2-cd4f32fa240d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.010430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a958f-aea9-4221-b1b2-cd4f32fa240d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.013020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:03.037524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbc52a-69d7-4acb-99f9-f5acfaa13123 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.072475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f84900-b3b1-4324-a743-c4b1da840b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.073092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f84900-b3b1-4324-a743-c4b1da840b53 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.073399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f84900-b3b1-4324-a743-c4b1da840b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.073922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f84900-b3b1-4324-a743-c4b1da840b53 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.074597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f84900-b3b1-4324-a743-c4b1da840b53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.076005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f84900-b3b1-4324-a743-c4b1da840b53 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.081657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a958f-aea9-4221-b1b2-cd4f32fa240d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.083094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a958f-aea9-4221-b1b2-cd4f32fa240d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.097059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 277/523] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:24:02 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:03.097924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f84900-b3b1-4324-a743-c4b1da840b53 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.139611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:03.203813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 247/524] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:03.204928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a958f-aea9-4221-b1b2-cd4f32fa240d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.223518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f84900-b3b1-4324-a743-c4b1da840b53 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.249973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:03.262543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.263198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.263482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.263785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.275908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.277396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:03.291885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:03.292818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:24:03.293765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:03.294342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 278/525] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:03.294733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.337263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a958f-aea9-4221-b1b2-cd4f32fa240d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.374463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01205e-460c-4135-983b-5785941763fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.375477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01205e-460c-4135-983b-5785941763fc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.375477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01205e-460c-4135-983b-5785941763fc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.375648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01205e-460c-4135-983b-5785941763fc None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.376253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01205e-460c-4135-983b-5785941763fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:03.376557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01205e-460c-4135-983b-5785941763fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.378497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01205e-460c-4135-983b-5785941763fc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.397271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 248/526] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:03.401809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:03.402866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01205e-460c-4135-983b-5785941763fc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.409677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc71cc3-3ac6-4ceb-9c49-b2d61b91f5fe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.452030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.453491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.453491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.453778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.454420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:03.454729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.456869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.475114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 279/527] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:03.476235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.511726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:03.519753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01205e-460c-4135-983b-5785941763fc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.556708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.557381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.557662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.557932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.569062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.570585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.587542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea95ea8-7c35-494e-9d56-e5f6327fd24d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:03.588027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:03.590935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:03.591805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:03.591805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:03.591805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:03.591805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:03.591805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 249/528] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:03.591805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.629274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.629834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.630118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.630385 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.631180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:03.631480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.632907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.650716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 280/529] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:03.651525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.667917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:03.727009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1801b87-61e8-4c04-b137-434e877a5e4d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.759489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ef567-d040-43e8-a6ec-c450677bb5a8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:03.762299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.762567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.762844 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.763115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.763891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=9f558687eefe4a26beb7d69151221660) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:03.764196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.765646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.781332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 250/530] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/role_assignments?user.id=9f558687eefe4a26beb7d69151221660&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:03.781332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.796242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:03.810044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:03.810935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:03.811335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:03.811630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:03.812423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=649334bb7ed04bd69eb971fc070d96e1) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:03.812900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:03.815978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:03.824046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 281/531] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/role_assignments?user.id=649334bb7ed04bd69eb971fc070d96e1&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:03.824909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:03.959549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fcfff-0b48-4e83-853a-bb2f4b15981f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:04.000504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:04.008503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:04.009151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:04.009462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:04.009736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:04.010437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:04.010742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:04.012115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:04.028623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 251/532] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:04.031843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bffa4-c9f1-42cb-9be7-29427f081e95 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:04.035455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:04.077986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb1a2b-35ed-4241-9f84-6877229ed3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:04.078783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb1a2b-35ed-4241-9f84-6877229ed3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:04.079572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb1a2b-35ed-4241-9f84-6877229ed3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:04.079854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb1a2b-35ed-4241-9f84-6877229ed3f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:04.118723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:04.170984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55a5e8-53d8-4b89-89bb-d9df6d4bfc44 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:04.207954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:04.208624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:04.208976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:04.209261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:04.210081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb1a2b-35ed-4241-9f84-6877229ed3f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:04.211677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb1a2b-35ed-4241-9f84-6877229ed3f5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:04.216610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:04.221328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:04.232387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:04.233508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:04.239126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 252/533] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:04.239594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:04.348354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:04.369534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15511e-2908-4421-8ff9-9ab3757aceb4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:04.372900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 282/534] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:04.382270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb1a2b-35ed-4241-9f84-6877229ed3f5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:04.401093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:04.408010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:04.408728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:04.409082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:04.409396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:04.410531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:04.411004 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:04.413619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:04.434429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 253/535] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:04.446811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:04.499017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb1a2b-35ed-4241-9f84-6877229ed3f5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:04.542576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:04.543585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:04.544086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:04.544550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:04.555620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:04.557388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:04.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:04.575466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:04.576314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:04.576314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:04.576314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:04.576314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:04.578538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 283/536] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:04.579941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd3743-d5f9-455a-aff8-82e48c180f9e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:04.590347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:04.617897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:04.618672 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:04.619408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:04.619809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:04.621002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=4b46a15d481f4393bef8c4fa2281710b) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:04.621444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:04.623126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:04.630385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 254/537] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:04 2022] GET /identity/v3/role_assignments?user.id=4b46a15d481f4393bef8c4fa2281710b&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:04.657711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:04.781932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc8c9a-2a9e-473a-ab60-3c350b350fb6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:04.820970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:04.821869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:04.822297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:04.822731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:04.823564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:04.824254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:04.825792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:04.839029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 284/538] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:04.844217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:04.905069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:04.967203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b9bdd-84b5-47ee-8259-cd0536292284 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:05.014702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:05.015581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:05.016065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:05.016475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:05.027781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:05.029398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:05.041138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:05.042173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:05.043205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:05.043205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:05.043205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:05.043205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:05.043205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:05.043205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:05.043205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:05.043205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:05.045216 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 255/539] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:05.054368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:05.170586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:05.181000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34397f-ff4c-40b0-be17-01561babdb95 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:05.220733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313780f2-5105-4b06-826b-6c6455f18f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:05.221584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313780f2-5105-4b06-826b-6c6455f18f4a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:05.221874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313780f2-5105-4b06-826b-6c6455f18f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:05.222154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313780f2-5105-4b06-826b-6c6455f18f4a None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:05.222908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313780f2-5105-4b06-826b-6c6455f18f4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:05.223218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313780f2-5105-4b06-826b-6c6455f18f4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:05.224900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313780f2-5105-4b06-826b-6c6455f18f4a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:05.238052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 285/540] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:05.243494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:05.370313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8509b4-5132-41de-b739-15a52bcc4c79 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:05.405652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b758c-c849-4a3b-811e-f3022858928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:05.407226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b758c-c849-4a3b-811e-f3022858928d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:05.407521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b758c-c849-4a3b-811e-f3022858928d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:05.407803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b758c-c849-4a3b-811e-f3022858928d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:05.409212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:05.410070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b758c-c849-4a3b-811e-f3022858928d None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=d6ee12d4109f4d8ba0d502558b9a2ea6) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:05.410612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b758c-c849-4a3b-811e-f3022858928d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:05.412203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b758c-c849-4a3b-811e-f3022858928d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:05.418650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 256/541] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:05 2022] GET /identity/v3/role_assignments?user.id=d6ee12d4109f4d8ba0d502558b9a2ea6&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:05.538167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe37f-7853-4129-bd4f-e575b3676fb3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:05.538694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe37f-7853-4129-bd4f-e575b3676fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:05.539036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe37f-7853-4129-bd4f-e575b3676fb3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:05.541128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 286/542] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:05.546305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef1939-bcd6-4f3b-b317-50ea792c345a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:05.546798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef1939-bcd6-4f3b-b317-50ea792c345a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:05.547224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef1939-bcd6-4f3b-b317-50ea792c345a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:05.574652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5cef1939-bcd6-4f3b-b317-50ea792c345a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:05.576202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5cef1939-bcd6-4f3b-b317-50ea792c345a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:05.665472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 257/543] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:05.701123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe37f-7853-4129-bd4f-e575b3676fb3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:05.842438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe37f-7853-4129-bd4f-e575b3676fb3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:05.888399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440b28c-a166-4992-866f-d68bd7eb8b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:05.889107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440b28c-a166-4992-866f-d68bd7eb8b99 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:05.889410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440b28c-a166-4992-866f-d68bd7eb8b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:05.889683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440b28c-a166-4992-866f-d68bd7eb8b99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:05.903657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440b28c-a166-4992-866f-d68bd7eb8b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9440b28c-a166-4992-866f-d68bd7eb8b99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:05.909461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:05.910386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:05.911195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:05.915204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 287/544] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:05.916527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef1939-bcd6-4f3b-b317-50ea792c345a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:06.067901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef1939-bcd6-4f3b-b317-50ea792c345a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:06.079876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:06.116894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:06.117946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.118455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.118930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.119859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:06.120363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:06.121989 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:06.131831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 258/545] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:06.137611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440b28c-a166-4992-866f-d68bd7eb8b99 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:06.164721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:06.299208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440b28c-a166-4992-866f-d68bd7eb8b99 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:06.351648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f87a8b-62c3-4910-9673-72a414574b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:06.352659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f87a8b-62c3-4910-9673-72a414574b70 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.353144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f87a8b-62c3-4910-9673-72a414574b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.353640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f87a8b-62c3-4910-9673-72a414574b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.368047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f87a8b-62c3-4910-9673-72a414574b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:06.369851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f87a8b-62c3-4910-9673-72a414574b70 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-96f87a8b-62c3-4910-9673-72a414574b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:06.397321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:06.398234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:06.399093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:06.399093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:06.399093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:06.399093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:06.401077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 288/546] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:06.406694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:06.454737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650dd8a-4151-4c24-a4d0-3361d64e9a12 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.455068 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650dd8a-4151-4c24-a4d0-3361d64e9a12 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.456134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650dd8a-4151-4c24-a4d0-3361d64e9a12 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.461522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 289/547] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:06.463211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ffb2-9972-4ea2-b20d-42ffb891fa3f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.463672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ffb2-9972-4ea2-b20d-42ffb891fa3f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.464118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ffb2-9972-4ea2-b20d-42ffb891fa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.485876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4781ffb2-9972-4ea2-b20d-42ffb891fa3f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:06.488828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4781ffb2-9972-4ea2-b20d-42ffb891fa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:06.590886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8427667-74f5-4bed-b77a-cc2bd2d21f02 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:06.599456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:06.614261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 290/548] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:06.616367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798a56e-bf7c-43ed-8617-341ce7a04568 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.616864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798a56e-bf7c-43ed-8617-341ce7a04568 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.617657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798a56e-bf7c-43ed-8617-341ce7a04568 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.619758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 291/549] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:06.625264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ab7e9-5e8a-4671-a2c0-0600b83dee16 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.625830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ab7e9-5e8a-4671-a2c0-0600b83dee16 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.626347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ab7e9-5e8a-4671-a2c0-0600b83dee16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.637810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:06.638895 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.639328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.639751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.640689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:06.641155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:06.643358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:06.657157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-123ab7e9-5e8a-4671-a2c0-0600b83dee16 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:06.658002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 259/550] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:06.659523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-123ab7e9-5e8a-4671-a2c0-0600b83dee16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:06.659776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:06.690704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:06.741326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 292/551] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:06.741326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ab7e9-5e8a-4671-a2c0-0600b83dee16 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:06.787763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846fc6-0f99-40b3-b4b7-d93678396a8a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:06.824852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f3efc-15e8-49ea-927e-b938a95697c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:06.825658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f3efc-15e8-49ea-927e-b938a95697c4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.826003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f3efc-15e8-49ea-927e-b938a95697c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.826365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f3efc-15e8-49ea-927e-b938a95697c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.837776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f3efc-15e8-49ea-927e-b938a95697c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-f05f3efc-15e8-49ea-927e-b938a95697c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:06.842945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:06.844427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 260/552] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:06.845369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f3efc-15e8-49ea-927e-b938a95697c4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:06.861374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ab7e9-5e8a-4671-a2c0-0600b83dee16 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:06.888520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:06.901650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:06.902621 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:06.902738 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:06.902970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:06.910949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:06.912401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:06.925350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:06.926353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:06.927457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 293/553] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:06.928043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:06.962773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:06.974563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f3efc-15e8-49ea-927e-b938a95697c4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.016030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f3053-0744-4509-b2bb-7ce0d039473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.017043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3053-0744-4509-b2bb-7ce0d039473f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.018117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3053-0744-4509-b2bb-7ce0d039473f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.018644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3053-0744-4509-b2bb-7ce0d039473f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.030434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f3053-0744-4509-b2bb-7ce0d039473f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ce3f3053-0744-4509-b2bb-7ce0d039473f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:07.035965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:07.037080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:07.038043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:07.039906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bfb7d-9e3f-4fde-9dc3-88833b0d050e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.045052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 261/554] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:07.045052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f3053-0744-4509-b2bb-7ce0d039473f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.076933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e331050-b2ee-428a-a253-2e61c4a08ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.077588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331050-b2ee-428a-a253-2e61c4a08ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.077877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331050-b2ee-428a-a253-2e61c4a08ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.078367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331050-b2ee-428a-a253-2e61c4a08ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.078872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e331050-b2ee-428a-a253-2e61c4a08ee3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:07.079251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e331050-b2ee-428a-a253-2e61c4a08ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:07.081108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e331050-b2ee-428a-a253-2e61c4a08ee3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:07.094647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 294/555] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:07.094647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be2f5-2169-45a5-9b75-76be2c3d3cc8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.094647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be2f5-2169-45a5-9b75-76be2c3d3cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.094647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be2f5-2169-45a5-9b75-76be2c3d3cc8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.094647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 295/556] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:07.095584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2be2f5-2169-45a5-9b75-76be2c3d3cc8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.147622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:07.157527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f3053-0744-4509-b2bb-7ce0d039473f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.194285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.194906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.195188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.195464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.196251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:07.196557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:07.198031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:07.209639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2be2f5-2169-45a5-9b75-76be2c3d3cc8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.216925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 262/557] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:07.217784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.231428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:07.248502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d65e29-c523-4d1f-9d0b-158e48c077f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.249319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65e29-c523-4d1f-9d0b-158e48c077f4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.249744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65e29-c523-4d1f-9d0b-158e48c077f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.250008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65e29-c523-4d1f-9d0b-158e48c077f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.250719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d65e29-c523-4d1f-9d0b-158e48c077f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:07.251023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d65e29-c523-4d1f-9d0b-158e48c077f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:07.253232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d65e29-c523-4d1f-9d0b-158e48c077f4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:07.261503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 296/558] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:07.262661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813169-2836-4fbe-9999-85fe12d611aa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.262943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813169-2836-4fbe-9999-85fe12d611aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.263216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813169-2836-4fbe-9999-85fe12d611aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.280509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7b813169-2836-4fbe-9999-85fe12d611aa None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:07.281712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7b813169-2836-4fbe-9999-85fe12d611aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:07.329741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8b80-b839-4a03-8b37-330e4cc561a2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.354077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 297/559] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:07.354837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b813169-2836-4fbe-9999-85fe12d611aa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.373508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.374165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.374565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.374907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.385759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:07.387257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:07.403106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:07.404179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:07.405105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:07.405105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:07.405105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:07.405105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:07.405105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 263/560] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:07.405415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.420304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:07.475668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:07.493962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b813169-2836-4fbe-9999-85fe12d611aa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.521211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0393a-1453-48ad-90ff-d51d9db46fb3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.529927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647cb99-b3d7-4337-887a-c5fb12b2b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.530553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cb99-b3d7-4337-887a-c5fb12b2b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.530838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cb99-b3d7-4337-887a-c5fb12b2b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.531115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cb99-b3d7-4337-887a-c5fb12b2b8c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.557642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.558286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.558626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.558909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.571153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:07.572675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:07.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:07.591503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:07.592403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:07.592403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:07.592403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:07.592403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:07.597678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 264/561] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:07.597678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.604184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647cb99-b3d7-4337-887a-c5fb12b2b8c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:07.605612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647cb99-b3d7-4337-887a-c5fb12b2b8c6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:07.644281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:07.710269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5993ff4-58cb-4a68-a198-6ffbc2800fae None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.724782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:07.733719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 298/562] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:07.734855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647cb99-b3d7-4337-887a-c5fb12b2b8c6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.747742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090e5c8-d534-418a-bd35-0816f0346475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.748407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090e5c8-d534-418a-bd35-0816f0346475 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.748684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090e5c8-d534-418a-bd35-0816f0346475 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.748957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090e5c8-d534-418a-bd35-0816f0346475 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.759507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090e5c8-d534-418a-bd35-0816f0346475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c090e5c8-d534-418a-bd35-0816f0346475 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:07.765012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:07.766095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 265/563] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:07.767238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090e5c8-d534-418a-bd35-0816f0346475 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.850836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647cb99-b3d7-4337-887a-c5fb12b2b8c6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.874182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090e5c8-d534-418a-bd35-0816f0346475 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:07.887106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.887939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.888429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.888999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.889723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:07.890037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:07.891518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:07.894636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:07.910896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 299/564] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:07.910896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.912845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:07.913491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:07.913772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:07.914042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:07.914729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:07.915031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:07.916388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:07.928721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 266/565] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:07.929834 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:07.959421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.008720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a109719-f250-4661-9dc3-35de5827b6ea None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.028767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776c768-d06b-4e6f-88bd-31cfafb12764 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.042960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.043593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.043873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.044138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.054410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.055834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.062051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf12f81-0281-4468-a959-359a4967cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.062657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf12f81-0281-4468-a959-359a4967cd62 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.063034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf12f81-0281-4468-a959-359a4967cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.063311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf12f81-0281-4468-a959-359a4967cd62 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.064013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf12f81-0281-4468-a959-359a4967cd62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:08.064377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf12f81-0281-4468-a959-359a4967cd62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.066329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf12f81-0281-4468-a959-359a4967cd62 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:08.070408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:08.071296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:08.072191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:08.072191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:08.072191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:08.072191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:08.072191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 300/566] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:08.072481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.082941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 267/567] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:08.082941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5da77-687e-414e-9f31-5f6861ec42e6 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.082941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5da77-687e-414e-9f31-5f6861ec42e6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.082941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5da77-687e-414e-9f31-5f6861ec42e6 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.082941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 268/568] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:08.082941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5da77-687e-414e-9f31-5f6861ec42e6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.120372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.173129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b43f48-ca04-4ceb-82b9-c90b6de98d8b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.178551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.186499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5da77-687e-414e-9f31-5f6861ec42e6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.207827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.208458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.208734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.209005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.216369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.217829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.220259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.221154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.221154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.221475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.228737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:08.228851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:08.229951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:08.230896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 301/569] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:08.231507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.231578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:08.244024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:08.245176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 269/570] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:08.246364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.347103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480b3fa-8c03-4828-bd2e-8d9c557df2ef None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.354749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ae54e-14bf-44d3-819e-fccdcbf88580 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.363745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.381729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.382369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.382650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.382927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.388420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be230a0-9bf4-4c3b-8204-4d6932dc38d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.389025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be230a0-9bf4-4c3b-8204-4d6932dc38d9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.389397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be230a0-9bf4-4c3b-8204-4d6932dc38d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.389679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be230a0-9bf4-4c3b-8204-4d6932dc38d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.390361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be230a0-9bf4-4c3b-8204-4d6932dc38d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:08.390657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be230a0-9bf4-4c3b-8204-4d6932dc38d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.392003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be230a0-9bf4-4c3b-8204-4d6932dc38d9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.393956 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.395376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.403497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 302/571] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:08.404716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61771a-76e4-42f9-ad9d-9e35bdd0d1d4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.404998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61771a-76e4-42f9-ad9d-9e35bdd0d1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.405260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61771a-76e4-42f9-ad9d-9e35bdd0d1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:08.410311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:08.411338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:08.412278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:08.412278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:08.412278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:08.412278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:08.412278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 270/572] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:08.412278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.420959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.424397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-eb61771a-76e4-42f9-ad9d-9e35bdd0d1d4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:08.425560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-eb61771a-76e4-42f9-ad9d-9e35bdd0d1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:08.493184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 303/573] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:08.494491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61771a-76e4-42f9-ad9d-9e35bdd0d1d4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.518243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7507-2605-4cb1-9353-d360ad5a0121 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.552598 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.553209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.553512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.553786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.554561 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:08.554864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.556273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.572941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 271/574] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:08.574111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.609364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.614878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61771a-76e4-42f9-ad9d-9e35bdd0d1d4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.637524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.649740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.650356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.650638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.650920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.651703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:08.652011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.653452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.671556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 304/575] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:08.672170 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.680276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2163f-f4c5-48d2-833f-fecf0d1a4fa2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.713990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.714627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.714915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.715186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.722358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.723797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:08.734788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:08.735851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 272/576] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:08.737841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.771636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f705505-1d58-4f1b-ab8e-8b56f46b4ff5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.804897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.805545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.805831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.806105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.806793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:08.807099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.808453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.819467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 305/577] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:08.820200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.838309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.841305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa89bcc-5f48-47b7-897c-83ded6ab54a3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.864830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:08.875936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d912-730d-40d4-856d-a82998cc36cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.876559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d912-730d-40d4-856d-a82998cc36cb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.877497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d912-730d-40d4-856d-a82998cc36cb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.877778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d912-730d-40d4-856d-a82998cc36cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.893110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7d912-730d-40d4-856d-a82998cc36cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-91c7d912-730d-40d4-856d-a82998cc36cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:08.901230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:08.902365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:08.903226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 273/578] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:08.903945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d912-730d-40d4-856d-a82998cc36cb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:08.925048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397674b-bf12-4d4b-97a0-9b94e227fd13 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:08.960788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:08.961395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:08.961681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:08.961961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:08.973255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:08.974719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:08.988734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:08.989854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:08.990771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:08.990771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:08.990771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:08.990771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:08.990771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 306/579] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:08.990771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.006458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d912-730d-40d4-856d-a82998cc36cb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.042589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.043218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.043498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.043767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.054258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.055706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.059616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:09.069938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:09.070971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:09.071876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:09.071876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:09.071876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:09.071876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:09.071876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 274/580] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:09.071876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.096154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b636c9d-1520-452b-a18e-39e482b3383f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.106949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:09.129800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.130421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.130701 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.130973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.131723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:09.132030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.133463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.145856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 307/581] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:09.146647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.173284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55c04-3ba2-4cbf-802a-a221d040c363 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.209123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.209789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.210069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.210350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.217337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.218703 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:09.230475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:09.231486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:09.232435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 275/582] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:09.233054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.255698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803759-c5b7-41da-9290-f96d4c509179 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.281494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:09.290319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.290930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.291208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.291478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.292179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:09.292481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.293856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.301665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 308/583] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:09.302433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.327775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:09.334704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915446c-8334-4988-b2e6-f104c624836e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.368579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.369190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.369501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.369777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.370458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:09.370762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.372134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.385664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 276/584] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:09.386481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.407625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ecf59-6f1d-4445-a65d-fa22ae67e05a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.441266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.441912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.442192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.442466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.443150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:09.443456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.444825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.457955 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 309/585] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:09.458758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.491359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5a85-644c-4b6a-8f99-65377b359a62 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.508505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:09.525566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.526190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.526471 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.526746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.527520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=25dd5ceedc3040fdb75d9d24a78956a0) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:09.527829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.529283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.535193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 277/586] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/role_assignments?user.id=25dd5ceedc3040fdb75d9d24a78956a0&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:09.536159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.560436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:09.576208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffecbdc-6bc3-4e69-8d85-1a91fc7939dc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.616538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.617204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.617478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.617755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.618577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:09.618887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.620359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.644983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 310/587] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:09.644983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.669757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4ed10-5122-4c3c-ad73-1b7534439efc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.711047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.711804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.712082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.712357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.724916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.726405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:09.744135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:09.745065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:09.746023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:09.746023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:09.746023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:09.746023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:09.746023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 278/588] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:09.746023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.766335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8a035-b4b9-45ee-b262-e098f5670d2e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.776299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:09.803647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.804283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.804573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.804846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.812100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.813753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:09.826321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:09.827276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:09.828162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 311/589] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:09.828755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.831974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:09.866705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e06fd3-ff57-406d-8984-1a32380ae4b8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.903940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.904606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.904891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.905169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.912249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.913748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:09.926368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:09.927299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:09.928184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 279/590] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:09.928671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:09.939082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db819f-4cbf-4246-8805-cfd4e9e6165b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:09.974801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32531956-4238-4402-8a00-34422f221e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:09.975435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32531956-4238-4402-8a00-34422f221e35 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:09.975718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32531956-4238-4402-8a00-34422f221e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:09.975980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32531956-4238-4402-8a00-34422f221e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:09.988845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32531956-4238-4402-8a00-34422f221e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:09.990326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32531956-4238-4402-8a00-34422f221e35 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-32531956-4238-4402-8a00-34422f221e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:10.005916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:10.006841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:10.007766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:10.007766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:10.007766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:10.007766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:10.007766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 312/591] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:10.007766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32531956-4238-4402-8a00-34422f221e35 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.034546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:10.040752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8868b-939b-4e94-aa8b-ebf5fd8b1b2a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.072632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:10.079504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.080694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.080971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.081239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.081963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:10.082264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.083629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.095649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 280/592] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:10.096445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.127898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32531956-4238-4402-8a00-34422f221e35 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.163740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.164375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.164636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.164904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.165784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:10.165966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.167395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.179661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 313/593] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:10.180275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.217397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b7129-80d9-4e0a-b8e0-cce3aa117d5e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.254612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.255230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.255510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.255781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.256572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:10.256820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.258284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.270760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:10.271267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 281/594] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:10.271963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.296884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58dcdc-0f79-4c2f-8638-380813dbed2f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.333121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.333865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.334188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.334499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.335120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:10.335422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.336933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.340928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:10.353655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 314/595] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:10.354805 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.397236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c0cbc-b939-41b0-81ab-237466d8d1fc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.443045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.445478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.445478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.445478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.459710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.461618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:10.477750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:10.478967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 282/596] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:10.480408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.493754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3759-308e-4d76-ae01-35f04ab4d1c7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.533455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef0923-2977-4df1-8f3a-8d8541f8b372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.534140 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0923-2977-4df1-8f3a-8d8541f8b372 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.534817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0923-2977-4df1-8f3a-8d8541f8b372 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.535407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0923-2977-4df1-8f3a-8d8541f8b372 None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/9f558687eefe4a26beb7d69151221660/roles/e6fe0c4d4ed1454f935b5c9001151260` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.549445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:10.556963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef0923-2977-4df1-8f3a-8d8541f8b372 None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=9f558687eefe4a26beb7d69151221660, role_id=e6fe0c4d4ed1454f935b5c9001151260)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.560280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef0923-2977-4df1-8f3a-8d8541f8b372 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.586740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 315/597] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:10 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/9f558687eefe4a26beb7d69151221660/roles/e6fe0c4d4ed1454f935b5c9001151260 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:10.586740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef0923-2977-4df1-8f3a-8d8541f8b372 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.602822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfe23d-0f32-4881-9db9-d8ec8384c8e8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.619096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:10.647783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110591a-ad05-4839-b812-3142ef2641cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.648636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110591a-ad05-4839-b812-3142ef2641cf None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.649016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110591a-ad05-4839-b812-3142ef2641cf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.649427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110591a-ad05-4839-b812-3142ef2641cf None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/649334bb7ed04bd69eb971fc070d96e1/roles/e6fe0c4d4ed1454f935b5c9001151260` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.667749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110591a-ad05-4839-b812-3142ef2641cf None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=649334bb7ed04bd69eb971fc070d96e1, role_id=e6fe0c4d4ed1454f935b5c9001151260)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.669493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110591a-ad05-4839-b812-3142ef2641cf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.692783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 283/598] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:10 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/649334bb7ed04bd69eb971fc070d96e1/roles/e6fe0c4d4ed1454f935b5c9001151260 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:10.694038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110591a-ad05-4839-b812-3142ef2641cf None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.777242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef0923-2977-4df1-8f3a-8d8541f8b372 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.832589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.833747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.834700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.835377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.841387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110591a-ad05-4839-b812-3142ef2641cf None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:10.846837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.848541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:10.866029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:10.867202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:10.868299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:10.868299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:10.868299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:10.868299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:10.868299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:10.868299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:10.868299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:10.868299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:10.871160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 316/599] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:10.877041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.894726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:10.895522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:10.895908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:10.896287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:10.897207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:10.897653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:10.899202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:10.925720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 284/600] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jul 28 06:24:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:10.931307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:10.966194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:10.967794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:11.038907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e7973-8dda-4e3f-96d4-6e3f922536c4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:11.067799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d4efb-27d7-4d34-9b4c-653a3306a4e3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:11.085121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:11.085851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:11.086240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:11.087328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:11.088206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:11.088628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:11.090217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:11.104318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 317/601] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:11.111666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:11.191111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:11.191780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:11.192067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:11.192342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:11.229517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07102dc1-a2b0-4b43-a7d7-dd9d04e2fd49 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:11.234183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:11.235674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:11.242021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:11.271348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:11.272495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:11.273601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:11.273601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:11.273601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:11.273601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:11.277063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 285/602] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:11.283772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:11.289006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a523a-9332-4c44-8623-00a125861506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:11.289906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a523a-9332-4c44-8623-00a125861506 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:11.293456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a523a-9332-4c44-8623-00a125861506 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:11.293894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a523a-9332-4c44-8623-00a125861506 None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/4b46a15d481f4393bef8c4fa2281710b/roles/e6fe0c4d4ed1454f935b5c9001151260` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:11.324977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a523a-9332-4c44-8623-00a125861506 None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=4b46a15d481f4393bef8c4fa2281710b, role_id=e6fe0c4d4ed1454f935b5c9001151260)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:11.326760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a523a-9332-4c44-8623-00a125861506 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:11.351590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 318/603] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:11 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/4b46a15d481f4393bef8c4fa2281710b/roles/e6fe0c4d4ed1454f935b5c9001151260 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:24:11.406718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:11.453105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596fdfc-c407-4c7f-a60f-f368e6778dad None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:11.502502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50757fec-775a-4203-9380-74b002447a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:11.503181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50757fec-775a-4203-9380-74b002447a89 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:11.503466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50757fec-775a-4203-9380-74b002447a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:11.503738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50757fec-775a-4203-9380-74b002447a89 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:11.504464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50757fec-775a-4203-9380-74b002447a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:11.504736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50757fec-775a-4203-9380-74b002447a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:11.506175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50757fec-775a-4203-9380-74b002447a89 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:11.520592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 286/604] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:11.526001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a523a-9332-4c44-8623-00a125861506 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:11.656677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a523a-9332-4c44-8623-00a125861506 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:11.700145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec28365-3d49-491c-bd5a-9d2b8d44c364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:11.701089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec28365-3d49-491c-bd5a-9d2b8d44c364 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:11.701551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec28365-3d49-491c-bd5a-9d2b8d44c364 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:11.702003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec28365-3d49-491c-bd5a-9d2b8d44c364 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:11.709798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec28365-3d49-491c-bd5a-9d2b8d44c364 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:11.711436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec28365-3d49-491c-bd5a-9d2b8d44c364 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-eec28365-3d49-491c-bd5a-9d2b8d44c364 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:11.726861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:11.727969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:11.729107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:11.729107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:11.729107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:11.729107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:11.729107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:11.729107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:11.729107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:11.729107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:11.731178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 319/605] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:11.737695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50757fec-775a-4203-9380-74b002447a89 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:11.852076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50757fec-775a-4203-9380-74b002447a89 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:11.859969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cd28f-3987-44fc-865b-ffe3a872b235 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:11.860491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cd28f-3987-44fc-865b-ffe3a872b235 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:11.860573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cd28f-3987-44fc-865b-ffe3a872b235 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:11.863111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 320/606] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:24:11.868314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391d595-b4af-4962-a83c-dbf93442be37 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:11.868607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391d595-b4af-4962-a83c-dbf93442be37 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:11.868885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391d595-b4af-4962-a83c-dbf93442be37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:11.887590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:11.894135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:11.895093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:11.895598 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:11.896072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:11.897102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:11.897785 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:11.899474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:11.910439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0391d595-b4af-4962-a83c-dbf93442be37 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:11.911718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0391d595-b4af-4962-a83c-dbf93442be37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:11.913634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 287/607] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:11.925447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:11.937657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:12.034848 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 321/608] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:12.069093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391d595-b4af-4962-a83c-dbf93442be37 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:12.099545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b37ee-4511-4cb5-96b4-04503836f742 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:12.171439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f21d7-7e52-4ba6-baa4-8fcd387478de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:12.172047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f21d7-7e52-4ba6-baa4-8fcd387478de None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:12.172329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f21d7-7e52-4ba6-baa4-8fcd387478de None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:12.172605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f21d7-7e52-4ba6-baa4-8fcd387478de None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/d6ee12d4109f4d8ba0d502558b9a2ea6/roles/e6fe0c4d4ed1454f935b5c9001151260` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:12.212092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f21d7-7e52-4ba6-baa4-8fcd387478de None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=d6ee12d4109f4d8ba0d502558b9a2ea6, role_id=e6fe0c4d4ed1454f935b5c9001151260)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:12.213585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f21d7-7e52-4ba6-baa4-8fcd387478de None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:12.231199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 288/609] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:11 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/d6ee12d4109f4d8ba0d502558b9a2ea6/roles/e6fe0c4d4ed1454f935b5c9001151260 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:12.259378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391d595-b4af-4962-a83c-dbf93442be37 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:12.318159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d104bdc-3293-41d1-93d7-4ee1ee0dab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:12.319275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104bdc-3293-41d1-93d7-4ee1ee0dab0c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:12.319761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104bdc-3293-41d1-93d7-4ee1ee0dab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:12.320234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104bdc-3293-41d1-93d7-4ee1ee0dab0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:12.338385 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d104bdc-3293-41d1-93d7-4ee1ee0dab0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0d104bdc-3293-41d1-93d7-4ee1ee0dab0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:12.343826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:12.344912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:12.346573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:12.349464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 322/610] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:12.355651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f21d7-7e52-4ba6-baa4-8fcd387478de None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:12.488049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:12.515478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f21d7-7e52-4ba6-baa4-8fcd387478de None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:12.552648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:12.553339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:12.553634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:12.553962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:12.554689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:12.554996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:12.556382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:12.565797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 289/611] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:12.574967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d104bdc-3293-41d1-93d7-4ee1ee0dab0c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:12.597489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:12.738104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d104bdc-3293-41d1-93d7-4ee1ee0dab0c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:12.792301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8606746-0b5b-46ca-a1ac-d786631a8330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:12.792946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8606746-0b5b-46ca-a1ac-d786631a8330 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:12.793224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8606746-0b5b-46ca-a1ac-d786631a8330 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:12.793518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8606746-0b5b-46ca-a1ac-d786631a8330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:12.808920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8606746-0b5b-46ca-a1ac-d786631a8330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:12.810386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8606746-0b5b-46ca-a1ac-d786631a8330 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a8606746-0b5b-46ca-a1ac-d786631a8330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:12.827022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:12.828007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:12.828956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:12.828956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:12.828956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:12.828956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:12.832349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 323/612] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:12.838114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:13.007976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd0121-974b-4ec6-8e7d-71e125d9eac1 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.008926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd0121-974b-4ec6-8e7d-71e125d9eac1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.009451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd0121-974b-4ec6-8e7d-71e125d9eac1 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.011032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 324/613] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:13.016234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e7890-ffe4-461e-99c9-7b4b28ec50e5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.016684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e7890-ffe4-461e-99c9-7b4b28ec50e5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.016960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e7890-ffe4-461e-99c9-7b4b28ec50e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.038707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037100f7-55a5-43c5-b628-0d277d7d47b3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:13.045622 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:13.046760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6a5e7890-ffe4-461e-99c9-7b4b28ec50e5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:13.047939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6a5e7890-ffe4-461e-99c9-7b4b28ec50e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:13.108594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:13.109222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.109533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.109810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.110504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:13.110809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:13.112246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:13.140868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 290/614] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:13.146390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:13.166348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:13.186607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 325/615] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:24:13.233438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e7890-ffe4-461e-99c9-7b4b28ec50e5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:13.314881 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb28003-a242-4f8e-a177-481332026904 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:13.386827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:13.387740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.388022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.388973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.400145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e7890-ffe4-461e-99c9-7b4b28ec50e5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:13.405023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:13.411201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:13.427884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:13.428900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:13.429933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:13.431882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 291/616] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:13.436585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:13.465346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f6c90-5346-4f22-acc9-1e6c3293114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:13.466026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6c90-5346-4f22-acc9-1e6c3293114f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.466547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6c90-5346-4f22-acc9-1e6c3293114f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.466834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f6c90-5346-4f22-acc9-1e6c3293114f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.478690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:13.482747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f6c90-5346-4f22-acc9-1e6c3293114f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-129f6c90-5346-4f22-acc9-1e6c3293114f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:13.490711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:13.491736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:13.492600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 326/617] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:13.497430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14c2d8-7b1f-4f75-8f7f-9426b15d3b1e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.498099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14c2d8-7b1f-4f75-8f7f-9426b15d3b1e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.498584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14c2d8-7b1f-4f75-8f7f-9426b15d3b1e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.503090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 327/618] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:13.507174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14c2d8-7b1f-4f75-8f7f-9426b15d3b1e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:13.576746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af5f68-d195-414a-9665-470ec5671b94 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:13.582224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:13.627176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7805cd-91aa-4bdc-9a4e-e267ad85a212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:13.627176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7805cd-91aa-4bdc-9a4e-e267ad85a212 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.627176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7805cd-91aa-4bdc-9a4e-e267ad85a212 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.627176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7805cd-91aa-4bdc-9a4e-e267ad85a212 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.627664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7805cd-91aa-4bdc-9a4e-e267ad85a212 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:13.627664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7805cd-91aa-4bdc-9a4e-e267ad85a212 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:13.628556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7805cd-91aa-4bdc-9a4e-e267ad85a212 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:13.649029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14c2d8-7b1f-4f75-8f7f-9426b15d3b1e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:13.651911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 292/619] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jul 28 06:24:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:13.653325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b523-414c-400a-9a4c-6d9f44551bf3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.653610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b523-414c-400a-9a4c-6d9f44551bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.653882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b523-414c-400a-9a4c-6d9f44551bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.675049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e537b523-414c-400a-9a4c-6d9f44551bf3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:13.676257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e537b523-414c-400a-9a4c-6d9f44551bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:13.698263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:13.699513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.699800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.700076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.700801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:13.701109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:13.703278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:13.718629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 328/620] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:13.722189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:13.775366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 293/621] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:13.776259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537b523-414c-400a-9a4c-6d9f44551bf3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:13.791442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:13.863859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d937b80-eaa0-4c16-833f-973263b897ab None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:13.883633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:13.885660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537b523-414c-400a-9a4c-6d9f44551bf3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:13.908906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:13.909650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.909882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.910149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.921243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:13.922676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:13.922796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f542d-159f-4179-9d69-1a5e17b5156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:13.923314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f542d-159f-4179-9d69-1a5e17b5156c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.923697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f542d-159f-4179-9d69-1a5e17b5156c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.923968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f542d-159f-4179-9d69-1a5e17b5156c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.934729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f542d-159f-4179-9d69-1a5e17b5156c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:13.936148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f542d-159f-4179-9d69-1a5e17b5156c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:13.937361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:13.938241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:13.939665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:13.939665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:13.939665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:13.939665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:13.939665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 329/622] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:13.939976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e33f542d-159f-4179-9d69-1a5e17b5156c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:13.958074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:13.959032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:13.959933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:13.959933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:13.959933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:13.959933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:13.959933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 294/623] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:13.960734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d509a1-2a5c-4396-972c-8fa1fbf0f3cd None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:13.960734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d509a1-2a5c-4396-972c-8fa1fbf0f3cd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:13.960998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d509a1-2a5c-4396-972c-8fa1fbf0f3cd None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:13.966208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 295/624] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:13.966208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d509a1-2a5c-4396-972c-8fa1fbf0f3cd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.037691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:14.055984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650736c-8567-41d2-b810-d3dc8c1f532a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.083765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d509a1-2a5c-4396-972c-8fa1fbf0f3cd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.093090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb53da-17ff-4280-8b06-d6130f711167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.093765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb53da-17ff-4280-8b06-d6130f711167 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.094050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb53da-17ff-4280-8b06-d6130f711167 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.094328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb53da-17ff-4280-8b06-d6130f711167 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.105956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb53da-17ff-4280-8b06-d6130f711167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-21bb53da-17ff-4280-8b06-d6130f711167 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:14.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:14.116603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:14.117507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:14.118183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 330/625] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:14.118897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb53da-17ff-4280-8b06-d6130f711167 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.126122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77447f8a-3fde-4654-8731-172864b4c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.127959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77447f8a-3fde-4654-8731-172864b4c44c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.128392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77447f8a-3fde-4654-8731-172864b4c44c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.128649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77447f8a-3fde-4654-8731-172864b4c44c None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.129400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77447f8a-3fde-4654-8731-172864b4c44c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:14.129706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77447f8a-3fde-4654-8731-172864b4c44c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.131082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77447f8a-3fde-4654-8731-172864b4c44c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.144487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 296/626] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:14.146889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f59e5-9332-481d-a9ab-6489a364b910 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.147595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f59e5-9332-481d-a9ab-6489a364b910 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.148032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f59e5-9332-481d-a9ab-6489a364b910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.151150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:14.166549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f24f59e5-9332-481d-a9ab-6489a364b910 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:14.167731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f24f59e5-9332-481d-a9ab-6489a364b910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:14.233076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb53da-17ff-4280-8b06-d6130f711167 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.242563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 297/627] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:14.243602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f59e5-9332-481d-a9ab-6489a364b910 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.280355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.281080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.281395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.281675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.282381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:14.282690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.284088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.301190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 331/628] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:14.301190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.316495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:14.367964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f59e5-9332-481d-a9ab-6489a364b910 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.397562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:14.406019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.406648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.406933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.407210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.407953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:14.408259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.409646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.414379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb088f07-5937-4335-862c-1cce0a6027d1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.417780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 298/629] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:14.422371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.460491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.461142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.461460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.461740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.468826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.470743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:14.482595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:14.483527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:14.485000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 332/630] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:14.485552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.536109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c786b-55e9-44d3-93fc-ab9c6fc1b3aa None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.553237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:14.573154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbf7c1-9962-49b2-ab01-49e7c149e679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.574122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbf7c1-9962-49b2-ab01-49e7c149e679 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.574361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbf7c1-9962-49b2-ab01-49e7c149e679 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.574634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbf7c1-9962-49b2-ab01-49e7c149e679 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.586714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdbf7c1-9962-49b2-ab01-49e7c149e679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-dcdbf7c1-9962-49b2-ab01-49e7c149e679 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:14.592161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:14.593393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:14.594390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:14.595273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb32c2d-e899-4a58-b4be-72f0165f47b6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.595357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 299/631] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:14.595357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbf7c1-9962-49b2-ab01-49e7c149e679 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.629540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.630097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.630380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.630654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.638232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.639789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:14.651806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:14.652888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:14.653915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 333/632] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:14.654523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.662898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:14.700187 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbf7c1-9962-49b2-ab01-49e7c149e679 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.735810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3248a3-076f-43df-b863-4e8bcae64917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.737258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3248a3-076f-43df-b863-4e8bcae64917 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.737579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3248a3-076f-43df-b863-4e8bcae64917 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.737862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3248a3-076f-43df-b863-4e8bcae64917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.748204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3248a3-076f-43df-b863-4e8bcae64917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.749686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3248a3-076f-43df-b863-4e8bcae64917 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.753220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96daa6-9245-4a85-8fc1-563d2f808081 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3e3248a3-076f-43df-b863-4e8bcae64917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:14.763686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:14.764583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:14.765509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:14.765509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:14.765509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:14.765509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:14.765509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 300/633] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:14.766010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9d2d-7f17-4b2d-8886-4105713d165c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.766264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9d2d-7f17-4b2d-8886-4105713d165c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.766550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9d2d-7f17-4b2d-8886-4105713d165c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.768282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 301/634] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:14.768962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b9d2d-7f17-4b2d-8886-4105713d165c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.791583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.792199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.792481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.792752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.793526 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:14.793830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.795245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.796462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:14.808267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 334/635] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:14.809065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.869885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b9d2d-7f17-4b2d-8886-4105713d165c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.875940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:14.902871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.903421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.903705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.903982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.904697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:14.905006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.906410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.912860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356ae3-8fcd-4da6-9b41-be6dbd6906b8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:14.914734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 302/636] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:14.915515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:14.948469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8258b2-57ef-48e7-ba3b-793952a09a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:14.949377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8258b2-57ef-48e7-ba3b-793952a09a13 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.949666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8258b2-57ef-48e7-ba3b-793952a09a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.949944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8258b2-57ef-48e7-ba3b-793952a09a13 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.950741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8258b2-57ef-48e7-ba3b-793952a09a13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:14.951051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8258b2-57ef-48e7-ba3b-793952a09a13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:14.952461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8258b2-57ef-48e7-ba3b-793952a09a13 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:14.969630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 335/637] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:14.970675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17668560-9871-42f0-af5f-29009d9ad5b9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:14.970970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17668560-9871-42f0-af5f-29009d9ad5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:14.971266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17668560-9871-42f0-af5f-29009d9ad5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:14.988766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-17668560-9871-42f0-af5f-29009d9ad5b9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:14.989984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-17668560-9871-42f0-af5f-29009d9ad5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:15.008188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:15.018284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aca6ba-1ed2-4f19-b44a-602b2350b6b0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.052242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.052860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.053145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.053465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.054156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:15.054467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:15.055840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:15.056704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 336/638] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:15.057524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17668560-9871-42f0-af5f-29009d9ad5b9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.078420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 303/639] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:15.079096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.114052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:15.160279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17668560-9871-42f0-af5f-29009d9ad5b9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.178837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec75d-122b-41db-b19c-8f6502a17abf None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.193577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5276a-a662-4f1e-b52f-00478b3787cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.194242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5276a-a662-4f1e-b52f-00478b3787cd None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.194522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5276a-a662-4f1e-b52f-00478b3787cd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.194798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5276a-a662-4f1e-b52f-00478b3787cd None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/25dd5ceedc3040fdb75d9d24a78956a0/roles/e6fe0c4d4ed1454f935b5c9001151260` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.209797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5276a-a662-4f1e-b52f-00478b3787cd None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=25dd5ceedc3040fdb75d9d24a78956a0, role_id=e6fe0c4d4ed1454f935b5c9001151260)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:15.211209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5276a-a662-4f1e-b52f-00478b3787cd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:15.212055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.212689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.212966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.213238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.224358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 337/640] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:15 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/25dd5ceedc3040fdb75d9d24a78956a0/roles/e6fe0c4d4ed1454f935b5c9001151260 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:15.224626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:15.225150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5276a-a662-4f1e-b52f-00478b3787cd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.226159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:15.255099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:15.256125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:15.262302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:15.262302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:15.262302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:15.262302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:15.262302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 304/641] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:15.262302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.265427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:15.390058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dee28-0675-40fc-a2fb-c45d7b18e789 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.431274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0067c-4e94-4ee3-92be-7ce740478b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.431847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0067c-4e94-4ee3-92be-7ce740478b2d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.432130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0067c-4e94-4ee3-92be-7ce740478b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.432406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0067c-4e94-4ee3-92be-7ce740478b2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.443234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0067c-4e94-4ee3-92be-7ce740478b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-24b0067c-4e94-4ee3-92be-7ce740478b2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:15.451042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:15.452059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 305/642] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:15.452953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0067c-4e94-4ee3-92be-7ce740478b2d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.468459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5276a-a662-4f1e-b52f-00478b3787cd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.471518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:15.505226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7d4c-5ce0-41ec-be4a-3bd20ddc0787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.505878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7d4c-5ce0-41ec-be4a-3bd20ddc0787 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.506161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7d4c-5ce0-41ec-be4a-3bd20ddc0787 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.506431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7d4c-5ce0-41ec-be4a-3bd20ddc0787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.525960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:15.569762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0067c-4e94-4ee3-92be-7ce740478b2d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.577754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc7d4c-5ce0-41ec-be4a-3bd20ddc0787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:15.579201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc7d4c-5ce0-41ec-be4a-3bd20ddc0787 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:15.606775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.607416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.607692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.608195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.615759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:15.617182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:15.629320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:15.630267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 306/643] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:15.631304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.691171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 338/644] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:15.692048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7d4c-5ce0-41ec-be4a-3bd20ddc0787 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.697841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:15.746780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801fb6a-fd0b-44ff-8389-b957912c6f97 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.770789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:15.783853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.784572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.784756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.785041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.786027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:15.786422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:15.787802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:15.804731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 307/645] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:15.804731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.808970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7d4c-5ce0-41ec-be4a-3bd20ddc0787 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.844854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.845600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.845913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.846189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.846898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:15.847202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:15.848549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:15.857106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 339/646] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:15.858470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:15.911925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529299d8-1e48-45ea-a0d8-db6cb0840401 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.932731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:15.947800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:15.948333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:15.948874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:15.949146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:15.949954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:15.950258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:15.951703 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:15.970918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 308/647] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:15.971825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5860c-7ed0-4961-86c8-796d6d41c3fd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:15.974143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.009405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.010052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.010328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.010596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.022729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.023284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.024512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:16.039897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:16.040874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:16.041814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:16.041814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:16.041814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:16.041814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:16.041814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 340/648] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:16.041814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.091486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed49fa4-53e3-47cb-8531-4bee182e762f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.129303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.130020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.130307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.130655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.137947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.139381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.148829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71c274-6f76-4bb0-9c9e-3ce94a5a0715 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:16.151627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:16.152565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 309/649] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:16.153565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.170090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.190914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.191547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.191826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.192094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.205625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.207304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:16.221919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:16.222826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:16.224268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:16.224268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:16.224268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:16.224268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:16.224268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 341/650] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:16.224268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.267786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.271111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc904c9a-c5d7-4905-995c-b1ce45aa4657 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.306047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f347c24-bef5-41f3-ab40-f1a84301f1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.306781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f347c24-bef5-41f3-ab40-f1a84301f1ea None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.307140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f347c24-bef5-41f3-ab40-f1a84301f1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.307419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f347c24-bef5-41f3-ab40-f1a84301f1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.335216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68a149-c316-4275-8fca-1f41ba7093fe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.373632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.374207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.374490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.374786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.375470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:16.375772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.377395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.382696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f347c24-bef5-41f3-ab40-f1a84301f1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.384150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f347c24-bef5-41f3-ab40-f1a84301f1ea None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.391016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 342/651] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:16.391745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.424222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.497777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.506344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 310/652] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:16.507289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f347c24-bef5-41f3-ab40-f1a84301f1ea None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.507411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cd652-a1e6-41f0-8245-5075442a5a2a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.543760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.544375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.544654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.544923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.545735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:16.546042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.547503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.566370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 343/653] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:16.571695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.610043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f347c24-bef5-41f3-ab40-f1a84301f1ea None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.645331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.645946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.646222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.646493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.647200 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:16.647500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.649423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.661644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.664048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 311/654] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:16.664782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.682134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd970a-d3b2-4be4-aa6e-853adccfdc64 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.717355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.718494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.718962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.719388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.727380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.728753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.730047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:16.743857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:16.744928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:16.746061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:16.746061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:16.746061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:16.746061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:16.746061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:16.746061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:16.746061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:16.746061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:16.753155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 344/655] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:16.754172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.774644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e776368-4a9f-4fa3-b0e4-732564591195 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.809527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.810166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.810454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.810714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.822344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.824040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:16.838360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:16.839310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:16.840232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:16.840232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:16.840232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:16.840232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:16.840232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 312/656] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:16.840232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:16.867808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8f5eb-3bac-4d0c-bf11-b7e318fc7598 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.904914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595160f0-7524-4d3b-b6e1-4484f8ad78ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.905561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595160f0-7524-4d3b-b6e1-4484f8ad78ac None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.905839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595160f0-7524-4d3b-b6e1-4484f8ad78ac None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.906105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595160f0-7524-4d3b-b6e1-4484f8ad78ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.926850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.947750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dcc26-2023-4707-9869-97bf38a90706 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:16.961570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:16.983557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595160f0-7524-4d3b-b6e1-4484f8ad78ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.985005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595160f0-7524-4d3b-b6e1-4484f8ad78ac None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:16.986566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:16.988835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:16.989145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:16.989452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:16.998442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:16.999880 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:17.012439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:17.013576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:24:17.014724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:17.016615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 313/657] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:17.018075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.102442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 345/658] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:17.103525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595160f0-7524-4d3b-b6e1-4484f8ad78ac None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.123049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa1838-164e-42a2-9b82-57d12c58e618 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:17.156326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:17.159819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:17.160473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.160753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.161024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.161807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:17.162114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:17.163528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.175379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 314/659] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:17.176483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.209742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:17.219104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595160f0-7524-4d3b-b6e1-4484f8ad78ac None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:17.255366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:17.255987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.256266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.256954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.257679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:17.257986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:17.260015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.276657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 346/660] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:17.276657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.287964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b50555-c741-433c-b875-8fd20db8452a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:17.322638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:17.323264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.323798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.324081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.324871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:17.325178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:17.326640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.343358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 315/661] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:17.344156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.378783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8d03-62fd-474c-be00-1e723f81a77a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:17.387219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:17.412096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:17.412722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.413000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.413273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.424233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:17.425677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.436922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:17.440357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:17.441281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:17.442321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:17.442321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:17.442321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:17.442321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:17.442321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 347/662] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:17.442604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.455065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdd312-756f-43ba-b4fd-0485464f662a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:17.489252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82037ad4-9a5a-4eeb-a980-aec719e554ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:17.489898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037ad4-9a5a-4eeb-a980-aec719e554ab None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.490183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037ad4-9a5a-4eeb-a980-aec719e554ab None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.490457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82037ad4-9a5a-4eeb-a980-aec719e554ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.491239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82037ad4-9a5a-4eeb-a980-aec719e554ab None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=9f558687eefe4a26beb7d69151221660) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:17.491548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82037ad4-9a5a-4eeb-a980-aec719e554ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:17.492973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82037ad4-9a5a-4eeb-a980-aec719e554ab None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.498938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 316/663] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/role_assignments?user.id=9f558687eefe4a26beb7d69151221660&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:17.499887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786df851-877c-4819-a9d9-66932877f7ad None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.500176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786df851-877c-4819-a9d9-66932877f7ad None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.500454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786df851-877c-4819-a9d9-66932877f7ad None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.505762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 317/664] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:17.505762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786df851-877c-4819-a9d9-66932877f7ad None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.557536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46f797-df14-46fd-9075-ca602a910338 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:17.596903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:17.597583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.597863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.598131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.605728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:17.607285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:17.619829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:17.620792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:17.621762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 348/665] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:17.622270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.638991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786df851-877c-4819-a9d9-66932877f7ad None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:17.647399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:17.684992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fea273-a3b5-4401-8a11-b0a07f4e8f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:17.685660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fea273-a3b5-4401-8a11-b0a07f4e8f18 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.685941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fea273-a3b5-4401-8a11-b0a07f4e8f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.686221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fea273-a3b5-4401-8a11-b0a07f4e8f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.721482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:17.752681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b2486-77be-4d12-8543-84c98cee00f5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:17.781836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fea273-a3b5-4401-8a11-b0a07f4e8f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:17.784874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fea273-a3b5-4401-8a11-b0a07f4e8f18 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.791374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8f0f6-ab93-4f85-8430-159fb86ef31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:17.792009 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8f0f6-ab93-4f85-8430-159fb86ef31a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.792292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8f0f6-ab93-4f85-8430-159fb86ef31a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.793037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8f0f6-ab93-4f85-8430-159fb86ef31a None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.793265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8f0f6-ab93-4f85-8430-159fb86ef31a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:17.793601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8f0f6-ab93-4f85-8430-159fb86ef31a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:17.795030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8f0f6-ab93-4f85-8430-159fb86ef31a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:17.809374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 349/666] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:17.810440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e90ef-d4ec-4c6e-bb29-93f9b98ed7c2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:17.810729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e90ef-d4ec-4c6e-bb29-93f9b98ed7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:17.811003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e90ef-d4ec-4c6e-bb29-93f9b98ed7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:17.831137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-358e90ef-d4ec-4c6e-bb29-93f9b98ed7c2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:17.832348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-358e90ef-d4ec-4c6e-bb29-93f9b98ed7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:17.910242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 350/667] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:17.911113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e90ef-d4ec-4c6e-bb29-93f9b98ed7c2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.913420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 318/668] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:17.914487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fea273-a3b5-4401-8a11-b0a07f4e8f18 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:17.938383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:17.985428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:18.035432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fea273-a3b5-4401-8a11-b0a07f4e8f18 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.046730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e90ef-d4ec-4c6e-bb29-93f9b98ed7c2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.075687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.076255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.076537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.076813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.084369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.085869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:18.086876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.087948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.088281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.089323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.090159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:18.090470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.092042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:18.098939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:18.099853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:18.100806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 319/669] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:18.101939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.108505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 351/670] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:18.113587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.201892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:18.232595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16f5a-d537-4d97-a8ed-ceca8bcdb412 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.235854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898b0ec-d1c6-4885-91d9-aa9617987e4f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.248777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:18.273749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e980-ce85-4f13-837e-9cd3f1e5258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.273996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.274401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e980-ce85-4f13-837e-9cd3f1e5258b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.274592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.274688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e980-ce85-4f13-837e-9cd3f1e5258b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.274857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.274954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e980-ce85-4f13-837e-9cd3f1e5258b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.275137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.286160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45e980-ce85-4f13-837e-9cd3f1e5258b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:18.287598 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.289229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0d45e980-ce85-4f13-837e-9cd3f1e5258b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:18.291874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:18.293018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:18.293925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 352/671] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:18.294650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e980-ce85-4f13-837e-9cd3f1e5258b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:18.309066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:18.310491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:18.311431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:18.311431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:18.311431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:18.311431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:18.316048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 320/672] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:18.316636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.424904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e980-ce85-4f13-837e-9cd3f1e5258b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.426517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2bce1-9d75-4b0f-b121-44dd00a82819 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.460302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:18.466698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.467702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.468085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.468461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.469341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:18.469753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.469848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.471926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.472203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.472402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:18.472507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.473251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=649334bb7ed04bd69eb971fc070d96e1) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:18.473584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.475020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:18.485976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 353/673] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:18.491580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.494009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 321/674] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/role_assignments?user.id=649334bb7ed04bd69eb971fc070d96e1&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:18.495074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.545566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:18.627951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172b8d-8a21-4dbf-943d-c79be1adb9d3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.632901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8c096-3e90-4610-b844-3e5fdaee19b7 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.668721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.672806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.672806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.672806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.672806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:18.672806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.672806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:18.673446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.674843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.675187 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.675408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.676104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:18.676410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.678018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:18.682034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 354/675] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:18.683056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.698817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 322/676] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:18.699045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.751211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:18.812580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bd267-2e1f-41ec-9ee0-315d5ab00da6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.836779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:18.849957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10ab53-1b2e-4727-8f78-6286969b438f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:18.872595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.873676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.874416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.875286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.876913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=4b46a15d481f4393bef8c4fa2281710b) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:18.877508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.879715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:18.889986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 355/677] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/role_assignments?user.id=4b46a15d481f4393bef8c4fa2281710b&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:18.891433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:18.923758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:18.924002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:18.924296 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:18.924555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:18.952742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:18.954304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:19.009213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:19.010656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:19.011558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:19.011558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:19.011558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:19.011558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:19.013634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 323/678] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:19.019849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:19.070673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbbcce-3d82-486c-b723-46dfa5105cea None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:19.087365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:19.114279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:19.115293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.115778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.116240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.123658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:19.125350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:19.140011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:19.140977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:19.141987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:19.141987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:19.141987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:19.141987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:19.141987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:19.141987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:19.141987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:19.141987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:19.142899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d923e4-f194-4897-9808-fc4293650970 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:19.144314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 356/679] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:19.150063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:19.185036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:19.186738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.187232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.187973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.189878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:19.190689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:19.192514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:19.208924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 324/680] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:19.214028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:19.217811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:19.280970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f365fb-a574-4f4d-ae34-6218266c141b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:19.318773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:19.319707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.320174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.320712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.321809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:19.322315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:19.324173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:19.338194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 357/681] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:19.339635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493ca6c-8cda-4435-88ff-4e182604fc47 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:19.346579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:19.384001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:19.384658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.385588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.385862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.403583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:19.405360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:19.405793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:19.430829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:19.431971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:19.432919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:19.435575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 325/682] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:19.446626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:19.483767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28de957-c4e3-46bc-9338-41fff5fcb32e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:19.527333 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:19.527957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.528235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.528510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.529322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=d6ee12d4109f4d8ba0d502558b9a2ea6) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:19.529629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:19.531048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:19.550537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 358/683] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:19 2022] GET /identity/v3/role_assignments?user.id=d6ee12d4109f4d8ba0d502558b9a2ea6&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:19.581793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:19.637143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acb6aa-623b-4ef8-8686-ef94635a972f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:19.684208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80315a4-c2fd-4a37-9e78-50b0bee1f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:19.685253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80315a4-c2fd-4a37-9e78-50b0bee1f03a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.685777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80315a4-c2fd-4a37-9e78-50b0bee1f03a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.686316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80315a4-c2fd-4a37-9e78-50b0bee1f03a None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.687431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80315a4-c2fd-4a37-9e78-50b0bee1f03a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:19.687948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80315a4-c2fd-4a37-9e78-50b0bee1f03a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:19.690253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80315a4-c2fd-4a37-9e78-50b0bee1f03a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:19.727265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 326/684] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jul 28 06:24:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:19.732646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:19.842669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:19.847114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cceece-3c4f-46e6-9b80-18f9ce41ec72 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:19.868273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1019c-8def-4969-a980-a26c6eb550cc None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.868596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1019c-8def-4969-a980-a26c6eb550cc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.868828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1019c-8def-4969-a980-a26c6eb550cc None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.870437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 327/685] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:19.876132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333ce83-f5f2-47e9-8f7d-2a2f06bccabc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.876563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333ce83-f5f2-47e9-8f7d-2a2f06bccabc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.877257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333ce83-f5f2-47e9-8f7d-2a2f06bccabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.893945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a95076-57c3-4eb6-bf6a-cf8740389c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:19.894927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a95076-57c3-4eb6-bf6a-cf8740389c91 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:19.895432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a95076-57c3-4eb6-bf6a-cf8740389c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:19.895938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a95076-57c3-4eb6-bf6a-cf8740389c91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:19.902902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8333ce83-f5f2-47e9-8f7d-2a2f06bccabc None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:19.904076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8333ce83-f5f2-47e9-8f7d-2a2f06bccabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:19.994250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 328/686] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 06:24:19.997665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a95076-57c3-4eb6-bf6a-cf8740389c91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:19.999299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a95076-57c3-4eb6-bf6a-cf8740389c91 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:20.017744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333ce83-f5f2-47e9-8f7d-2a2f06bccabc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:20.105882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:20.155377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:20.162074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 359/687] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:20.167792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a95076-57c3-4eb6-bf6a-cf8740389c91 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:20.227459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333ce83-f5f2-47e9-8f7d-2a2f06bccabc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:20.287123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a95076-57c3-4eb6-bf6a-cf8740389c91 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:20.327377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded84c-298d-4e7e-aa04-fd87a616c484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:20.327377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded84c-298d-4e7e-aa04-fd87a616c484 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:20.327377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded84c-298d-4e7e-aa04-fd87a616c484 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:20.327377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded84c-298d-4e7e-aa04-fd87a616c484 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:20.338566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ded84c-298d-4e7e-aa04-fd87a616c484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:20.342528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:20.343188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:20.343464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:20.343740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c3ded84c-298d-4e7e-aa04-fd87a616c484 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:20.344135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:20.345744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:20.346691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 329/688] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:20.354204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded84c-298d-4e7e-aa04-fd87a616c484 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:20.365828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:20.367209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:20.394601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:20.395760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:20.397429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:20.397429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:20.397429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:20.397429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:20.397429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 360/689] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:20.402335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:20.505287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:20.543126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded84c-298d-4e7e-aa04-fd87a616c484 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:20.558416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:20.587753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b18d30-3e18-4b4a-92a3-b03ab86979ab None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:20.607156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:20.607774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:20.608051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:20.608324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:20.609041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:20.609359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:20.610825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:20.635852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 330/690] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:20.642524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:20.656119 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:20.659317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:20.659594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:20.659864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:20.660565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:20.660862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:20.662251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:20.687030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 361/691] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:20.690104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:20.847076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf339e-75a1-4cc8-9a33-65e775d3f397 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:20.853808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3713-1059-456e-9b74-d68e4b37438b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:20.881674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:20.916130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:20.916697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:20.917470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:20.917749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:20.919699 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:20.920339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:20.920616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:20.920886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:20.938282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:20.939826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:20.943069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:20.944557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:20.964610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:20.965704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:20.966604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 362/692] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:20.967085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:20.967879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:20.968807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:20.968807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:20.968807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:20.968807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:20.968807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 331/693] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:20.974238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:20.974978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:21.015734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:21.139953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d7fb-56df-4786-8433-384b06e199af None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:21.144872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4592b9-c981-4614-ba95-93e3d986d74b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:21.176801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec591e3-d715-4b82-b799-23495abb812f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:21.177355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec591e3-d715-4b82-b799-23495abb812f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.177641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec591e3-d715-4b82-b799-23495abb812f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.177913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec591e3-d715-4b82-b799-23495abb812f None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.178664 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec591e3-d715-4b82-b799-23495abb812f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:21.178968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec591e3-d715-4b82-b799-23495abb812f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:21.180528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec591e3-d715-4b82-b799-23495abb812f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:21.184725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a186c9d-0c13-4aaa-bba9-880d96744a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:21.185711 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a186c9d-0c13-4aaa-bba9-880d96744a9f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.186204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a186c9d-0c13-4aaa-bba9-880d96744a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.186671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a186c9d-0c13-4aaa-bba9-880d96744a9f None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.187552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a186c9d-0c13-4aaa-bba9-880d96744a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:21.188063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a186c9d-0c13-4aaa-bba9-880d96744a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:21.190333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a186c9d-0c13-4aaa-bba9-880d96744a9f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:21.194586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 363/694] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:21.195837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ddc2-2ce7-45c4-92e0-eaf97279e7e0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.196123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ddc2-2ce7-45c4-92e0-eaf97279e7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.196399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687ddc2-2ce7-45c4-92e0-eaf97279e7e0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.202913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 364/695] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:21.207018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687ddc2-2ce7-45c4-92e0-eaf97279e7e0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:21.209198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 332/696] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:21.210670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f45b-2cbc-42b5-91ea-0f6e6245edb5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.211176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f45b-2cbc-42b5-91ea-0f6e6245edb5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.211644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f45b-2cbc-42b5-91ea-0f6e6245edb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.233836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ab56f45b-2cbc-42b5-91ea-0f6e6245edb5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:21.235385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ab56f45b-2cbc-42b5-91ea-0f6e6245edb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:21.237001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:21.313556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 333/697] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:21.315259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687ddc2-2ce7-45c4-92e0-eaf97279e7e0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:21.316068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56f45b-2cbc-42b5-91ea-0f6e6245edb5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:21.340822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:21.355602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:21.356221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.356557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.356835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.357672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=25dd5ceedc3040fdb75d9d24a78956a0) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:21.358059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:21.359439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:21.366745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 365/698] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:21 2022] GET /identity/v3/role_assignments?user.id=25dd5ceedc3040fdb75d9d24a78956a0&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:21.367540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:21.472188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56f45b-2cbc-42b5-91ea-0f6e6245edb5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:21.512430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2758d-b68c-4251-a3c0-b5c8a4b14750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:21.513095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2758d-b68c-4251-a3c0-b5c8a4b14750 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.513421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2758d-b68c-4251-a3c0-b5c8a4b14750 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.513699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2758d-b68c-4251-a3c0-b5c8a4b14750 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.522815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2758d-b68c-4251-a3c0-b5c8a4b14750 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:21.524291 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2758d-b68c-4251-a3c0-b5c8a4b14750 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2cf2758d-b68c-4251-a3c0-b5c8a4b14750 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:21.538813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:21.539849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:21.540802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 334/699] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:24:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:21.545791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94da6-1398-4d88-8cac-7e6fb9eb6746 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.546089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94da6-1398-4d88-8cac-7e6fb9eb6746 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.546379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94da6-1398-4d88-8cac-7e6fb9eb6746 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.551902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 335/700] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:21.551902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b94da6-1398-4d88-8cac-7e6fb9eb6746 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:21.563408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:21.573813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c5094-a6cf-477c-b2fb-50bc88428f66 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:21.617537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f423e3-e7e0-498e-a6fe-e2031feba57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:21.618184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f423e3-e7e0-498e-a6fe-e2031feba57c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.618470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f423e3-e7e0-498e-a6fe-e2031feba57c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.618746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f423e3-e7e0-498e-a6fe-e2031feba57c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.631970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f423e3-e7e0-498e-a6fe-e2031feba57c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:21.633456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f423e3-e7e0-498e-a6fe-e2031feba57c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-80f423e3-e7e0-498e-a6fe-e2031feba57c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:21.651117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:21.652055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:21.652959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:21.652959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:21.652959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:21.652959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:21.655174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 366/701] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:21.656761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09de19-9461-4c15-84fe-72f4a44d2cb4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.657267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09de19-9461-4c15-84fe-72f4a44d2cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.657848 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09de19-9461-4c15-84fe-72f4a44d2cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.666425 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:21.680312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ef09de19-9461-4c15-84fe-72f4a44d2cb4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:21.683380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ef09de19-9461-4c15-84fe-72f4a44d2cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:21.696228 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b94da6-1398-4d88-8cac-7e6fb9eb6746 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:21.737098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:21.737802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.738084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.738352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:21.739201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:21.739503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:21.740929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:21.760067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 336/702] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:24:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:21.760954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:21.773497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 367/703] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:21.774355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09de19-9461-4c15-84fe-72f4a44d2cb4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:21.862731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:21.887751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09de19-9461-4c15-84fe-72f4a44d2cb4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:21.916837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d104d30-d14e-444e-a23a-b2b334208649 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:21.927927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035295fc-c26e-44e3-a86f-badac49680d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:21.928560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035295fc-c26e-44e3-a86f-badac49680d6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:21.928842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035295fc-c26e-44e3-a86f-badac49680d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:21.929118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035295fc-c26e-44e3-a86f-badac49680d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.016592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035295fc-c26e-44e3-a86f-badac49680d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.018788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035295fc-c26e-44e3-a86f-badac49680d6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.096115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:22.101616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.102232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.102509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.102779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.103472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:22.103760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.105110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.107913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:22.118899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 337/704] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:22.119956 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.143344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 368/705] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:22.144713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6c4d-b3da-49b9-8826-c031881830d6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.145002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6c4d-b3da-49b9-8826-c031881830d6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.145279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6c4d-b3da-49b9-8826-c031881830d6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.146853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 369/706] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:22.147542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6c4d-b3da-49b9-8826-c031881830d6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.230292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf247d9-611a-42e8-9432-e96a2e05d6d6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:22.259512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6c4d-b3da-49b9-8826-c031881830d6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:22.265132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.265803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.266080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.266348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] PATH_INFO: `/v3/services/network` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.267057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.268576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:22.279774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:22.280883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:22.282417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:22.286986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 338/707] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:22 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:22.287737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.302515 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac3779-3d27-42ba-8448-752e8bd05a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.303149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac3779-3d27-42ba-8448-752e8bd05a5e None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.303468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac3779-3d27-42ba-8448-752e8bd05a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.303752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac3779-3d27-42ba-8448-752e8bd05a5e None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.304489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac3779-3d27-42ba-8448-752e8bd05a5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.305945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac3779-3d27-42ba-8448-752e8bd05a5e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.327952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 370/708] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:24:22 2022] POST /identity/v3/users => generated 254 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:22.330079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54281-6618-4f63-99a2-104e009cac8a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.330687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54281-6618-4f63-99a2-104e009cac8a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.331156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54281-6618-4f63-99a2-104e009cac8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.354631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a0d54281-6618-4f63-99a2-104e009cac8a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:22.355889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a0d54281-6618-4f63-99a2-104e009cac8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:22.370150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:22.391083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:22.438477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db8a56-579b-48b2-82a1-862e426216de None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:22.455469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 371/709] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:22.456341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d54281-6618-4f63-99a2-104e009cac8a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.488623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.490431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.490916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.491386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.504419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.506175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:22.521173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:22.522134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:22.523051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:22.523051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:22.523051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:22.523051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:22.523051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 339/710] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:22.524876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.596680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d54281-6618-4f63-99a2-104e009cac8a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:22.633512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.634145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.634431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.634705 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.635389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] RBAC: Adding query filter params (name=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:22.635697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.637057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.642821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 372/711] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:22.643683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.655908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa669f3-a48a-4cce-a499-296de3985810 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:22.674994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:22.678949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:22.704261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.705582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.705934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.706210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.706952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.708265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:22.717882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:22.718847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:22.719748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 340/712] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Jul 28 06:24:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:22.721902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.775223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d9c77-c03a-4857-bd76-d3ef7ac64b32 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:22.816446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.817099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.817447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.817648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.818340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:22.818646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.820003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.837267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 373/713] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:22.837267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.856903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187a675-105f-40ce-b876-8adce6310689 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:22.894075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.895259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.896031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.896777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.897731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] RBAC: Adding query filter params (type=network) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:22.898377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.900274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.909014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 341/714] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:22.910446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:22.931868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:22.948413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b3066-0927-4375-916e-02c49f7ba717 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:22.956324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:22.988164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:22.988932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:22.989282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:22.989753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:22.990493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:22.990881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:22.992422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:22.997695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 374/715] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jul 28 06:24:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:23.010587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:23.033858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920b948-b365-43c7-889c-cb538547bf92 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:23.090134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:23.091212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:23.091728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:23.092224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:23.100620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:23.109336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:23.133798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:23.134801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:23.135714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 342/716] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:23.141507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:23.160892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e71f58-ed77-4dea-a337-aeb4463c67d7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:23.221644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:23.222275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:23.222549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:23.223289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:23.223495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:23.223800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:23.225165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:23.232609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 375/717] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jul 28 06:24:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:23.288190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:23.296291 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de87baa-b9bd-4ce5-8369-c99d88f29857 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:23.360628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499ae52-6c9d-4811-b604-f24b25b3141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:23.361646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ae52-6c9d-4811-b604-f24b25b3141a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:23.362082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ae52-6c9d-4811-b604-f24b25b3141a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:23.362533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ae52-6c9d-4811-b604-f24b25b3141a None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:23.363515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499ae52-6c9d-4811-b604-f24b25b3141a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:23.363993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499ae52-6c9d-4811-b604-f24b25b3141a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:23.366220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499ae52-6c9d-4811-b604-f24b25b3141a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:23.386145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 343/718] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:23.396189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:23.419127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:23.513351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14982b69-419c-43ca-abdd-a0227643b686 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:23.553898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2b10a-417b-43c8-a26b-3fbf47720daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:23.554461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2b10a-417b-43c8-a26b-3fbf47720daf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:23.554744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2b10a-417b-43c8-a26b-3fbf47720daf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:23.555020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2b10a-417b-43c8-a26b-3fbf47720daf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:23.555820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2b10a-417b-43c8-a26b-3fbf47720daf None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=9f558687eefe4a26beb7d69151221660) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:23.556134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2b10a-417b-43c8-a26b-3fbf47720daf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:23.557591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2b10a-417b-43c8-a26b-3fbf47720daf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:23.564234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 376/719] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:23 2022] GET /identity/v3/role_assignments?user.id=9f558687eefe4a26beb7d69151221660&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:23.817129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b64b9-3655-4584-8243-f3e644101e26 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:23.817600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b64b9-3655-4584-8243-f3e644101e26 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:23.818100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b64b9-3655-4584-8243-f3e644101e26 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:23.820229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 344/720] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:23.835727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ac82f-5007-46fd-98f9-a1a4311a057a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:23.836131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ac82f-5007-46fd-98f9-a1a4311a057a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:23.836415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ac82f-5007-46fd-98f9-a1a4311a057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:23.855225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-880ac82f-5007-46fd-98f9-a1a4311a057a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:23.856423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-880ac82f-5007-46fd-98f9-a1a4311a057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:23.939927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 377/721] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:23.986871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b64b9-3655-4584-8243-f3e644101e26 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:24.125336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b64b9-3655-4584-8243-f3e644101e26 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:24.179503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac8689-76f1-4284-b0cf-0ceb86c9420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:24.180155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac8689-76f1-4284-b0cf-0ceb86c9420c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.180436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac8689-76f1-4284-b0cf-0ceb86c9420c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.180711 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac8689-76f1-4284-b0cf-0ceb86c9420c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.198196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac8689-76f1-4284-b0cf-0ceb86c9420c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8aac8689-76f1-4284-b0cf-0ceb86c9420c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:24.203402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:24.206310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:24.207891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 345/722] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:24.214647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ac82f-5007-46fd-98f9-a1a4311a057a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:24.234931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:24.343940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ac82f-5007-46fd-98f9-a1a4311a057a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:24.384824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:24.385403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.385690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.385967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.386700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:24.387006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:24.388385 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:24.397712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 378/723] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:24.404175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac8689-76f1-4284-b0cf-0ceb86c9420c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:24.443403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:24.565256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac8689-76f1-4284-b0cf-0ceb86c9420c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:24.614920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5542777-e20c-4096-af90-0854b599a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:24.615928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5542777-e20c-4096-af90-0854b599a1fe None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.616440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5542777-e20c-4096-af90-0854b599a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.616939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5542777-e20c-4096-af90-0854b599a1fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.630954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5542777-e20c-4096-af90-0854b599a1fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:24.632692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5542777-e20c-4096-af90-0854b599a1fe None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b5542777-e20c-4096-af90-0854b599a1fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:24.651211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:24.652125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:24.653114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:24.653114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:24.653114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:24.653114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:24.659704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 346/724] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:24.661315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:24.721649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9c57c-4d65-44b6-8ded-16fe5c492858 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.722231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9c57c-4d65-44b6-8ded-16fe5c492858 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.722688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9c57c-4d65-44b6-8ded-16fe5c492858 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.724525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 347/725] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:24.730138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30145a7-ee3f-48f3-9ebe-2e707515307d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.730423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30145a7-ee3f-48f3-9ebe-2e707515307d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.730700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30145a7-ee3f-48f3-9ebe-2e707515307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.766150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d30145a7-ee3f-48f3-9ebe-2e707515307d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:24.769051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d30145a7-ee3f-48f3-9ebe-2e707515307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:24.828414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:24.874309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:24.884321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 348/726] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:24.895254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d40f31-3a6c-4ae8-a6a0-ee4becd30539 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:24.908262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30145a7-ee3f-48f3-9ebe-2e707515307d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:24.963324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:24.964345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:24.964834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:24.965367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:24.966271 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:24.968288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:24.974904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:24.993441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 379/727] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:24.999060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.036928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30145a7-ee3f-48f3-9ebe-2e707515307d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.079530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc722c1-cdc4-4555-b373-93f5319c639a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.080186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc722c1-cdc4-4555-b373-93f5319c639a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.080469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc722c1-cdc4-4555-b373-93f5319c639a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.080745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc722c1-cdc4-4555-b373-93f5319c639a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.102207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc722c1-cdc4-4555-b373-93f5319c639a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-bdc722c1-cdc4-4555-b373-93f5319c639a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:25.107189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:25.108168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:25.109557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 349/728] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:25.116915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc722c1-cdc4-4555-b373-93f5319c639a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.158582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bcbdf-00ff-46fe-bc39-1fa99f0af72f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.193707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:25.196534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.197431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.197900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.198352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.214967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.217634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:25.240156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:25.241416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:25.242629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:25.244886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:25.247282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 380/729] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:25.255246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.319803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc722c1-cdc4-4555-b373-93f5319c639a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.396965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.397863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.398294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.398716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.399606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:25.400061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.401608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.421179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 350/730] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:25.426935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.445386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ec79-d5ff-4b4f-ad19-470567e01410 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.507290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbde9ce-eae0-48b1-a826-a4089938e859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.507924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbde9ce-eae0-48b1-a826-a4089938e859 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.508469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbde9ce-eae0-48b1-a826-a4089938e859 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.508749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbde9ce-eae0-48b1-a826-a4089938e859 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.510069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbde9ce-eae0-48b1-a826-a4089938e859 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:25.510662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbde9ce-eae0-48b1-a826-a4089938e859 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.512736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbde9ce-eae0-48b1-a826-a4089938e859 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.537694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 381/731] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jul 28 06:24:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:25.538972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6acaf-b9c1-400d-a467-bbe24dca820e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.539261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6acaf-b9c1-400d-a467-bbe24dca820e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.539537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6acaf-b9c1-400d-a467-bbe24dca820e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.545440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 382/732] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:25.546223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6acaf-b9c1-400d-a467-bbe24dca820e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.585148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06545d6-899f-4fde-ad37-1050cc095e81 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.609574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:25.615092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:25.634591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86880635-2697-420d-9154-7b32542ec7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.636006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86880635-2697-420d-9154-7b32542ec7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.636525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86880635-2697-420d-9154-7b32542ec7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.637038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86880635-2697-420d-9154-7b32542ec7e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.652232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86880635-2697-420d-9154-7b32542ec7e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.655100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86880635-2697-420d-9154-7b32542ec7e1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.663459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6acaf-b9c1-400d-a467-bbe24dca820e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-86880635-2697-420d-9154-7b32542ec7e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:25.674811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:25.675733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:25.676660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:25.676660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:25.676660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:25.676660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:25.684418 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 351/733] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:25.684418 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea34e4-aa02-4e7a-81d7-9e97bb772e9f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.684418 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea34e4-aa02-4e7a-81d7-9e97bb772e9f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.684418 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea34e4-aa02-4e7a-81d7-9e97bb772e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.705673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-55ea34e4-aa02-4e7a-81d7-9e97bb772e9f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:25.707467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-55ea34e4-aa02-4e7a-81d7-9e97bb772e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:25.709004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f7d28-513d-407c-a522-ea5fa442007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.709781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f7d28-513d-407c-a522-ea5fa442007b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.710202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f7d28-513d-407c-a522-ea5fa442007b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.710610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f7d28-513d-407c-a522-ea5fa442007b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.785285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f7d28-513d-407c-a522-ea5fa442007b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.786822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f7d28-513d-407c-a522-ea5fa442007b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.793474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 352/734] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:25.793474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea34e4-aa02-4e7a-81d7-9e97bb772e9f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.850620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:25.904738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea34e4-aa02-4e7a-81d7-9e97bb772e9f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:25.906274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:25.909362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 383/735] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 06:24:25.910615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b450246-8dfe-4edf-ac4c-2f69a0c5903b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.910905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b450246-8dfe-4edf-ac4c-2f69a0c5903b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.911185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b450246-8dfe-4edf-ac4c-2f69a0c5903b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.916393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 384/736] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:25.916393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b450246-8dfe-4edf-ac4c-2f69a0c5903b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:25.946867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:25.947401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:25.947680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:25.947950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:25.948891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:25.949192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:25.950571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:25.961994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 353/737] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:25.962791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.027364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b450246-8dfe-4edf-ac4c-2f69a0c5903b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.065927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca21993-10b0-4f9f-b9ba-564b05ebae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.066905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca21993-10b0-4f9f-b9ba-564b05ebae2e None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.067383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca21993-10b0-4f9f-b9ba-564b05ebae2e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.067839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca21993-10b0-4f9f-b9ba-564b05ebae2e None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.068211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f940a-2660-41be-8e8c-17860ef1162a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.068916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca21993-10b0-4f9f-b9ba-564b05ebae2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.070495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca21993-10b0-4f9f-b9ba-564b05ebae2e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.088808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 385/738] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:24:25 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:26.090836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188645-d957-434c-8b4e-c15a75ec1d39 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.091337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188645-d957-434c-8b4e-c15a75ec1d39 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.092060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188645-d957-434c-8b4e-c15a75ec1d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.120613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0d188645-d957-434c-8b4e-c15a75ec1d39 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:26.121651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.122299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.122579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.122851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.123199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0d188645-d957-434c-8b4e-c15a75ec1d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:26.128506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.139838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.141808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:26.160002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:26.161017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:26.161984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:26.161984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:26.161984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:26.161984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:26.161984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 354/739] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:26.162281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.176097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.219050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 386/740] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:26 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:26.221020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc929-2e7f-4413-a275-f2b4e0fe508a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.221605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc929-2e7f-4413-a275-f2b4e0fe508a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.222107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc929-2e7f-4413-a275-f2b4e0fe508a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.223650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 387/741] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:26.224558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dc929-2e7f-4413-a275-f2b4e0fe508a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.294146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738665d-f43e-4a0d-beda-f557cb677ff6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.328238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c554d92-d9a3-4b13-88ff-ddc57bfbaf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.328919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c554d92-d9a3-4b13-88ff-ddc57bfbaf52 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.329281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c554d92-d9a3-4b13-88ff-ddc57bfbaf52 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.330092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c554d92-d9a3-4b13-88ff-ddc57bfbaf52 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.337568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dc929-2e7f-4413-a275-f2b4e0fe508a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.338784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c554d92-d9a3-4b13-88ff-ddc57bfbaf52 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.340214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c554d92-d9a3-4b13-88ff-ddc57bfbaf52 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6c554d92-d9a3-4b13-88ff-ddc57bfbaf52 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:26.352854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:26.354017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:26.354990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 355/742] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:24:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:26.355612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f69ed4-75ee-4e78-8cff-8fe544a05cba None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.355792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f69ed4-75ee-4e78-8cff-8fe544a05cba None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.356118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f69ed4-75ee-4e78-8cff-8fe544a05cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.375043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b7f69ed4-75ee-4e78-8cff-8fe544a05cba None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:26.376865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b7f69ed4-75ee-4e78-8cff-8fe544a05cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:26.382275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.382993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.383347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.383692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.384464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:26.384835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.386300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.396063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.400551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 388/743] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:26.401472 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.454893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.460817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 356/744] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:26 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 06:24:26.464866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f69ed4-75ee-4e78-8cff-8fe544a05cba None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.515975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb5493-77a0-4c77-a97e-23c05ba1997d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.550734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335f0f-46cf-4e9c-9454-688a25edb352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.551354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335f0f-46cf-4e9c-9454-688a25edb352 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.551641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335f0f-46cf-4e9c-9454-688a25edb352 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.551914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335f0f-46cf-4e9c-9454-688a25edb352 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.552600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20335f0f-46cf-4e9c-9454-688a25edb352 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.554178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20335f0f-46cf-4e9c-9454-688a25edb352 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.577327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 389/745] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:24:26 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:26.577327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335f0f-46cf-4e9c-9454-688a25edb352 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.593518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f69ed4-75ee-4e78-8cff-8fe544a05cba None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.646972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.654428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.655062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.655782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.657082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.659176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:26.659488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.661992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.688282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 357/746] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jul 28 06:24:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:26.690756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.699970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335f0f-46cf-4e9c-9454-688a25edb352 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.739330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.739983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.740264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.740568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.747684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.749197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:26.762487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:26.763080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:26.763977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:26.765026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 390/747] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:26.765557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.825558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275a6dd-08fb-487d-9467-22832c9811e0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.872881 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b250b9-c88b-4f1d-a9e4-8569c3a0bd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.874181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b250b9-c88b-4f1d-a9e4-8569c3a0bd7c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.874990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b250b9-c88b-4f1d-a9e4-8569c3a0bd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.875729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b250b9-c88b-4f1d-a9e4-8569c3a0bd7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.883314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac839a-b7ba-42e9-ae4d-c4801ca128c3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:26.890171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b250b9-c88b-4f1d-a9e4-8569c3a0bd7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-55b250b9-c88b-4f1d-a9e4-8569c3a0bd7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:26.897019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:26.898230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:26.899270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:26.901743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 358/748] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:26.907797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b250b9-c88b-4f1d-a9e4-8569c3a0bd7c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:26.920610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763df9ba-c1fc-4840-b345-33473a88a600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:26.921238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763df9ba-c1fc-4840-b345-33473a88a600 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:26.921591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763df9ba-c1fc-4840-b345-33473a88a600 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:26.921918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763df9ba-c1fc-4840-b345-33473a88a600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:26.925635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:26.995071 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763df9ba-c1fc-4840-b345-33473a88a600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:26.996531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763df9ba-c1fc-4840-b345-33473a88a600 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.017239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b250b9-c88b-4f1d-a9e4-8569c3a0bd7c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:27.032951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.055120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:27.056061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.056558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.057051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.058023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:27.058528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:27.060142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.073101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 359/749] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:27.074491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.128077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 391/750] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:24:27.128931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763df9ba-c1fc-4840-b345-33473a88a600 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.193715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.206982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367cbbe5-3734-4af2-99c8-dd36b29f4d80 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:27.244245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:27.244867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.245152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.245479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.246396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:27.246713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:27.248130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.264589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 360/751] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:27.264589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.280981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763df9ba-c1fc-4840-b345-33473a88a600 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:27.301482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.331763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:27.332394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.332793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.333069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.334148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=25dd5ceedc3040fdb75d9d24a78956a0) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:27.334464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:27.336571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.344647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 392/752] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/role_assignments?user.id=25dd5ceedc3040fdb75d9d24a78956a0&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:27.345813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.405559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220374f-61bc-4f7a-8f8d-9a8c9da11959 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:27.469272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:27.471571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.472155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.472656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.485340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:27.488358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:27.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:27.507729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:27.508652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:27.508652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:27.508652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:27.508652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:27.511102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 361/753] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:27.519044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.540555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.563159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a001f-6284-480a-b2a4-16b4a34b6ea6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:27.609384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:27.610248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.611180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.611553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.623614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:27.625193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:27.641027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:27.642439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:27.643381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:27.643381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:27.643381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:27.643381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:27.644191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 393/754] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:27.649057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.677100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.700728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b888-3b88-4742-a558-ec9dea227703 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:27.764201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dafdd-62dd-4a5d-b8b6-0e518012d9f9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:27.802909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:27.803131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:27.803561 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.803851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.804129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.804448 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:27.804836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:27.805029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:27.805155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:27.805593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:27.806433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:27.806545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.806976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:27.808621 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:27.811408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:27.824353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 394/755] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:27.832148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.835133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 362/756] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:27.840329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:27.966079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd675a-01c9-4fd5-b509-f20d8bd87dd8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.007626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.008284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.008568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.008843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.017199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.019275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:28.032040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:28.033271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:28.034392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:28.034392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:28.034392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:28.034392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:28.034392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:28.034392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:28.034392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:28.034392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:28.039057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 363/757] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:28.039918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.053814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:28.092797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a3d-0329-47d4-aa9e-2f522801b8aa None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.138062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.138766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.139054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.139380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.147927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.149401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:28.162498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:28.164347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:28.165575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 395/758] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:24:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:28.169412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.195204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97ad21-d4a1-411d-ba50-8f02424ab72e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.237606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:28.242932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58fe86-1b8c-411c-8fe1-12606678b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.243566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58fe86-1b8c-411c-8fe1-12606678b692 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.243848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58fe86-1b8c-411c-8fe1-12606678b692 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.244121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58fe86-1b8c-411c-8fe1-12606678b692 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.245472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58fe86-1b8c-411c-8fe1-12606678b692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:28.245819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58fe86-1b8c-411c-8fe1-12606678b692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.247333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58fe86-1b8c-411c-8fe1-12606678b692 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.262394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 364/759] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:28.263755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf046-8e37-4ed0-9eb0-7dd56ae05f86 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.264047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf046-8e37-4ed0-9eb0-7dd56ae05f86 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.264323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf046-8e37-4ed0-9eb0-7dd56ae05f86 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.269375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 365/760] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:28.270124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cf046-8e37-4ed0-9eb0-7dd56ae05f86 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.294814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104ad5d-dd8c-420c-9eea-355c227d14ce None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.335344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054ec31-cf02-4a02-99e7-c864b8faba2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.336067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ec31-cf02-4a02-99e7-c864b8faba2a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.336353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ec31-cf02-4a02-99e7-c864b8faba2a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.336624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ec31-cf02-4a02-99e7-c864b8faba2a None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.337508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054ec31-cf02-4a02-99e7-c864b8faba2a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:28.337853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054ec31-cf02-4a02-99e7-c864b8faba2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.339489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054ec31-cf02-4a02-99e7-c864b8faba2a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.346910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:28.366148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 396/761] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:24:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:28.367693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582c987-4493-4ec2-ac9f-a35fc35e5e05 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.367989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582c987-4493-4ec2-ac9f-a35fc35e5e05 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.368266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582c987-4493-4ec2-ac9f-a35fc35e5e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.387129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cf046-8e37-4ed0-9eb0-7dd56ae05f86 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.395781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6582c987-4493-4ec2-ac9f-a35fc35e5e05 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:28.397025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6582c987-4493-4ec2-ac9f-a35fc35e5e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:28.437124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.437793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.438077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.439591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.440391 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=528c54678648461397f490fbbb698601) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:28.440701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.442163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.453883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 366/762] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:28 2022] GET /identity/v3/role_assignments?user.id=528c54678648461397f490fbbb698601&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:28.454638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.487391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 397/763] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:28.510631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582c987-4493-4ec2-ac9f-a35fc35e5e05 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.563039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:28.599598 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bd4cc-48d9-4551-a04e-ed737be0ac78 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.633704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:28.637328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582c987-4493-4ec2-ac9f-a35fc35e5e05 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.641693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.642201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.642615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.643046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.654924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.656527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:28.674382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:28.675434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:28.676636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:28.676636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:28.676636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:28.676636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:28.676636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 367/764] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:28.681869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.684677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.685725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.687082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.687587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] PATH_INFO: `/v3/services/network` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.688423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.690848 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:28.712416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:28.713521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:28.714409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:28.715943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 398/765] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:28 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:28.725589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.817893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33029626-e2dd-4972-80f7-8ed14825a191 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.854240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:28.855137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:28.855417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:28.855692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:28.856540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:28.856846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:28.858231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:28.872026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 368/766] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:28.877868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:28.887893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9003683-95e0-45d8-9e2c-2ff356c5d8fa None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:28.909036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:28.929678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:28.988866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e577-3a15-49ca-98c5-c9c898698177 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:29.006603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:29.007244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.007527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.007810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.008498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] RBAC: Adding query filter params (name=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:29.008806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:29.010198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:29.025790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:29.026630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 399/767] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:29.026856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.027279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.027694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.032836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:29.035892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:29.037583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:29.051397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:29.052503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:29.053640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:29.053640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:29.053640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:29.053640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:29.053640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:29.053640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:29.053640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:29.053640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:29.056129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 369/768] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:29.066871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:29.171673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd41eae-1ff6-4d38-899b-098f6500ecda None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:29.211115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bd0a0-edd7-4545-bc12-452cff655bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:29.212382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bd0a0-edd7-4545-bc12-452cff655bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.212876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bd0a0-edd7-4545-bc12-452cff655bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.213522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bd0a0-edd7-4545-bc12-452cff655bf8 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.214718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bd0a0-edd7-4545-bc12-452cff655bf8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:29.215262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bd0a0-edd7-4545-bc12-452cff655bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:29.217375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bd0a0-edd7-4545-bc12-452cff655bf8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:29.224465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 400/769] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:29.225991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e79a9-72f9-4b77-807c-e7d7ae80cd5d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.226493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e79a9-72f9-4b77-807c-e7d7ae80cd5d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.227008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e79a9-72f9-4b77-807c-e7d7ae80cd5d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.232817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 401/770] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:29.234188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e79a9-72f9-4b77-807c-e7d7ae80cd5d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:29.238860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bc05d-db4b-4347-97bc-8646a1d09696 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:29.242253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:29.281474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:29.285149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454089d1-f133-4d5f-9932-89199c84f976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:29.285931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454089d1-f133-4d5f-9932-89199c84f976 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.286317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454089d1-f133-4d5f-9932-89199c84f976 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.286814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454089d1-f133-4d5f-9932-89199c84f976 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.287682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454089d1-f133-4d5f-9932-89199c84f976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:29.288081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454089d1-f133-4d5f-9932-89199c84f976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:29.289632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454089d1-f133-4d5f-9932-89199c84f976 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:29.309368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 370/771] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:29.310944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a382-f3c0-4dde-b372-6760aac85aca None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.311238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a382-f3c0-4dde-b372-6760aac85aca None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.311515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a382-f3c0-4dde-b372-6760aac85aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.331921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8c10a382-f3c0-4dde-b372-6760aac85aca None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:29.333217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8c10a382-f3c0-4dde-b372-6760aac85aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:29.367842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e79a9-72f9-4b77-807c-e7d7ae80cd5d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:29.408401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:29.409494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.410050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.410595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.411560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] RBAC: Adding query filter params (interface=public, service_id=65aa4d604afd4009a6dc42adf212d710, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:29.412069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:29.413722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:29.415760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 371/772] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:29.416616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10a382-f3c0-4dde-b372-6760aac85aca None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:29.428252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 402/773] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jul 28 06:24:29 2022] GET /identity/v3/endpoints?service_id=65aa4d604afd4009a6dc42adf212d710&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:29.429474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:29.542416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:29.551806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10a382-f3c0-4dde-b372-6760aac85aca None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:29.565710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:29.571436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250ef7a-8218-4504-a7bb-3ec8fd02de2a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:29.591086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79c87d-31fa-4de3-87ef-1c773e12abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:29.592088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c87d-31fa-4de3-87ef-1c773e12abfc None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.592523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c87d-31fa-4de3-87ef-1c773e12abfc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.592992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c87d-31fa-4de3-87ef-1c773e12abfc None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/9f558687eefe4a26beb7d69151221660/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.595903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79c87d-31fa-4de3-87ef-1c773e12abfc None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=9f558687eefe4a26beb7d69151221660, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:29.597565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79c87d-31fa-4de3-87ef-1c773e12abfc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:29.615021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 372/774] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:29 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/9f558687eefe4a26beb7d69151221660/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:29.616526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79c87d-31fa-4de3-87ef-1c773e12abfc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:29.622469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d7322-cf9d-4840-80c7-9a1000c7626a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:29.623241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d7322-cf9d-4840-80c7-9a1000c7626a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.623645 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d7322-cf9d-4840-80c7-9a1000c7626a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.623992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d7322-cf9d-4840-80c7-9a1000c7626a None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.624732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d7322-cf9d-4840-80c7-9a1000c7626a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:29.626298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d7322-cf9d-4840-80c7-9a1000c7626a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:29.637554 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 403/775] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:24:29 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:29.714078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea2dc6-6264-4fde-b6fb-1f17697b36b0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.714272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea2dc6-6264-4fde-b6fb-1f17697b36b0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.714851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea2dc6-6264-4fde-b6fb-1f17697b36b0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.717457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 404/776] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:29.722799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2543b-4730-4cbd-a183-600b816134d3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.722799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2543b-4730-4cbd-a183-600b816134d3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.723095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2543b-4730-4cbd-a183-600b816134d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.738642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79c87d-31fa-4de3-87ef-1c773e12abfc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:29.778730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6ef2543b-4730-4cbd-a183-600b816134d3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:29.779705 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ef2543b-4730-4cbd-a183-600b816134d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:29.785005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:29.785991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:29.787573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:29.788045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:29.788873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:29.790486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:29.802833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:29.804066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:29.805147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:29.808381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 373/777] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Jul 28 06:24:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:29.814483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:29.853948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:29.936038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 405/778] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:29.981906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0662a8-b942-4932-8edc-2ad5fd85371c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:29.986952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2543b-4730-4cbd-a183-600b816134d3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:30.018554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:30.019186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.019472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.019749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.020432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:30.020737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:30.022131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:30.027233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 374/779] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jul 28 06:24:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:30.033222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:30.133750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:30.143968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18498280-b707-4072-9079-0cdfaa09cee8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:30.186258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:30.196682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2543b-4730-4cbd-a183-600b816134d3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:30.207467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:30.208100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.208376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.208650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.209343 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:30.209652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:30.211015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:30.218854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 375/780] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jul 28 06:24:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:30.226061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:30.272632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d7332-d582-49e3-be3a-1ac6c80e1fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:30.273204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7332-d582-49e3-be3a-1ac6c80e1fba None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.274896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7332-d582-49e3-be3a-1ac6c80e1fba None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.275201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7332-d582-49e3-be3a-1ac6c80e1fba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.306239 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d7332-d582-49e3-be3a-1ac6c80e1fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-cf5d7332-d582-49e3-be3a-1ac6c80e1fba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:30.311413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:30.312367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:30.313231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:30.321755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 406/781] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:30.334343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d7332-d582-49e3-be3a-1ac6c80e1fba None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:30.355617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab4da1-466e-451e-81cd-18c87bf2aed9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:30.392855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:30.393506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.393789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.394060 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.395258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] RBAC: Adding query filter params (interface=public, service_id=df0196e6884c4b52933b80fb6882888d, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:30.395616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:30.397024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:30.405395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 376/782] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jul 28 06:24:30 2022] GET /identity/v3/endpoints?service_id=df0196e6884c4b52933b80fb6882888d&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:30.411573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:30.445677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:30.535000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59207137-3d47-48d4-85e9-783dc0366a0b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:30.573105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16480c84-33c1-4ecc-b01f-b61bbedaf561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:30.574142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480c84-33c1-4ecc-b01f-b61bbedaf561 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.574563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480c84-33c1-4ecc-b01f-b61bbedaf561 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.574841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480c84-33c1-4ecc-b01f-b61bbedaf561 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.575493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16480c84-33c1-4ecc-b01f-b61bbedaf561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:30.576943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16480c84-33c1-4ecc-b01f-b61bbedaf561 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:30.582845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 377/783] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:24:30 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:30.593661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d7332-d582-49e3-be3a-1ac6c80e1fba None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:30.650566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:30.651480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.651912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.652335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.653233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:30.653745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:30.655312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:30.670982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 407/784] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:30.676832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16480c84-33c1-4ecc-b01f-b61bbedaf561 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:30.784441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:30.809425 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16480c84-33c1-4ecc-b01f-b61bbedaf561 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:30.846014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e04c8-ac32-468b-b6da-c78b7a4fe317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:30.846659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e04c8-ac32-468b-b6da-c78b7a4fe317 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.846980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e04c8-ac32-468b-b6da-c78b7a4fe317 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.847222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e04c8-ac32-468b-b6da-c78b7a4fe317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.858012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e04c8-ac32-468b-b6da-c78b7a4fe317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:30.859453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e04c8-ac32-468b-b6da-c78b7a4fe317 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-0b1e04c8-ac32-468b-b6da-c78b7a4fe317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:30.874075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:30.875017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:30.877070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:30.877070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:30.877070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:30.877070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:30.877070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 378/785] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:30.881366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:30.905181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:30.933760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a72bd4-81e8-449b-b49e-d1c04837dd4e None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.934341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a72bd4-81e8-449b-b49e-d1c04837dd4e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.934620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a72bd4-81e8-449b-b49e-d1c04837dd4e None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.936122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 379/786] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:24:30.941973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17e592-6955-49d4-8f61-232d74c5c3d7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:30.942260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17e592-6955-49d4-8f61-232d74c5c3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:30.942538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17e592-6955-49d4-8f61-232d74c5c3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:30.965695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bc17e592-6955-49d4-8f61-232d74c5c3d7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:30.966882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bc17e592-6955-49d4-8f61-232d74c5c3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:31.031506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdebaa-75ca-4987-9a6a-32a8fe5c58f8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:31.078438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 380/787] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:31.095456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:31.096659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:31.097209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:31.097724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:31.098687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:31.099199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:31.100830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:31.122436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 408/788] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:31.128897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17e592-6955-49d4-8f61-232d74c5c3d7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:31.133192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:31.228497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:31.264610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:31.301748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17e592-6955-49d4-8f61-232d74c5c3d7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:31.306102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f17051-4564-4ae5-bfda-90a353a6405f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:31.363264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:31.364791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:31.366568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:31.367630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:31.370027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e241ac6-6151-4853-8bd4-23b9b1c09b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:31.371912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e241ac6-6151-4853-8bd4-23b9b1c09b9f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:31.372606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e241ac6-6151-4853-8bd4-23b9b1c09b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:31.373755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e241ac6-6151-4853-8bd4-23b9b1c09b9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:31.381899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:31.387975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:31.404945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e241ac6-6151-4853-8bd4-23b9b1c09b9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0e241ac6-6151-4853-8bd4-23b9b1c09b9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:31.410509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:31.411464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:31.412322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:31.414905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 409/789] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:31.420871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:31.421858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:31.422988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:31.429361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e241ac6-6151-4853-8bd4-23b9b1c09b9f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:31.429607 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 381/790] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:31.437072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:31.606644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5cd2e-f403-409e-918a-a6980794d693 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:31.633350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:31.642971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e241ac6-6151-4853-8bd4-23b9b1c09b9f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:31.671438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:31.673866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:31.677148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:31.678118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:31.679134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:31.680589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:31.683507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:31.701682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:31.702650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:31.703269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:31.703770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:31.704767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:31.705278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:31.706993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:31.709704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:31.718585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 410/791] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:31.725343 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 382/792] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jul 28 06:24:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:31.726624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:31.729438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:31.839876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c6332-ad3b-4405-b847-4ff548c135cf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:31.877138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:31.878268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:31.878810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:31.879302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:31.892490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:31.894288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:31.910875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:31.911829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:31.912733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:31.912733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:31.912733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:31.912733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:31.919198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de296e-a8b3-4c66-966f-2feee09e5ff6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:31.919583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 411/793] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:31.921074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:31.978622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:31.983596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:31.984250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:31.984529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:31.984797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.007819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.009425 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.035943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f920-f2f2-4c16-8354-fb28f0c6b8c4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:32.036586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:32.037525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:32.043305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:32.043305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:32.043305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:32.043305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:32.043305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 383/794] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:32.045107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.095159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.096295 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.096816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.097313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.098252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:32.098783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.100453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.117470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 412/795] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:32.118713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:32.120376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.198892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8284935-69cc-4c74-900c-f4f573a6e524 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.234156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff72d75-e841-4ff3-9d3b-dacc68be0c92 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.235191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7743dc-e4fd-4668-a63e-e364881934c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.236183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7743dc-e4fd-4668-a63e-e364881934c5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.237168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7743dc-e4fd-4668-a63e-e364881934c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.237168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7743dc-e4fd-4668-a63e-e364881934c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.237464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7743dc-e4fd-4668-a63e-e364881934c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:32.237774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7743dc-e4fd-4668-a63e-e364881934c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.239138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7743dc-e4fd-4668-a63e-e364881934c5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.245411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:32.255645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 384/796] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:32.257288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304933-04d3-4efd-a4f8-f169a8bf2821 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.258207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304933-04d3-4efd-a4f8-f169a8bf2821 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.258716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304933-04d3-4efd-a4f8-f169a8bf2821 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.260684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 385/797] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:32.262314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf058ce-6529-49e6-a55a-2653766416f2 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.262599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf058ce-6529-49e6-a55a-2653766416f2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.262876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf058ce-6529-49e6-a55a-2653766416f2 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.264231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 386/798] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:32.264770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf058ce-6529-49e6-a55a-2653766416f2 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.276308 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b2565-cbd9-45d6-bdd1-572f7a755ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.276956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b2565-cbd9-45d6-bdd1-572f7a755ffe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.277237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b2565-cbd9-45d6-bdd1-572f7a755ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.277539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b2565-cbd9-45d6-bdd1-572f7a755ffe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.285654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b2565-cbd9-45d6-bdd1-572f7a755ffe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.287225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b2565-cbd9-45d6-bdd1-572f7a755ffe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-694b2565-cbd9-45d6-bdd1-572f7a755ffe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:32.301473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:32.302477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:32.303428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 413/799] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:32.304037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13336aa2-c796-4626-aa3b-9bf1dad88233 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.304037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13336aa2-c796-4626-aa3b-9bf1dad88233 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.304710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13336aa2-c796-4626-aa3b-9bf1dad88233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.326359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-13336aa2-c796-4626-aa3b-9bf1dad88233 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:32.327552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-13336aa2-c796-4626-aa3b-9bf1dad88233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:32.365502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:32.385795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf058ce-6529-49e6-a55a-2653766416f2 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.405332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 414/800] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:32.406352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86afff4-e566-4696-8346-3a730940675a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.406644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86afff4-e566-4696-8346-3a730940675a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.406917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86afff4-e566-4696-8346-3a730940675a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.433060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c86afff4-e566-4696-8346-3a730940675a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:32.434368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c86afff4-e566-4696-8346-3a730940675a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:32.440304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.440952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.441236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.441540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.448407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.450294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:32.462928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:32.463839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 387/801] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:32.465682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.516125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 415/802] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:32.521585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86afff4-e566-4696-8346-3a730940675a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.548294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:32.586273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cae20-48e7-404f-978e-3986ff30059d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.622444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.623057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.623366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.623604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.624377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:32.624815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.626190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.630740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86afff4-e566-4696-8346-3a730940675a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.651432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 388/803] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:32.651432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.655684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:32.671880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b013df-378b-415b-9973-07b539bfe6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.672615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b013df-378b-415b-9973-07b539bfe6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.673400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b013df-378b-415b-9973-07b539bfe6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.673872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b013df-378b-415b-9973-07b539bfe6e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.685238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b013df-378b-415b-9973-07b539bfe6e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a7b013df-378b-415b-9973-07b539bfe6e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:32.690869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:32.691810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:32.692659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 416/804] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:32.693392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b013df-378b-415b-9973-07b539bfe6e1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.764729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18978dda-9b68-4f0b-ad81-b70366cffa40 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.790149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:32.802783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b013df-378b-415b-9973-07b539bfe6e1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.806170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.806787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.807067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.807342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.808111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:32.808416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.809870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.822278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 389/805] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:32.823334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.841756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.842476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.842751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.842948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] PATH_INFO: `/v3/services/network` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.843559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.845617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:32.854527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:32.855565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:32.857042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 417/806] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:32.858234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:32.901415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:32.945906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93101f7-a7e0-4236-a706-80a8bbd2aea3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.964799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976f33b-4ba0-43ee-8146-3f71cf52222f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:32.985975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.986619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.987078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:32.987350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:32.998008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:32.998892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:32.999435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:32.999533 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:32.999791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.000070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.001318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:33.001946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.002986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.011190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 418/807] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:33.011963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:33.014297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:33.015399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:33.016491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:33.016491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:33.016491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:33.016491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:33.016491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 390/808] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:33.016491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.037501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:33.118103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ab689-d0a9-4d78-bb23-2c931482bb26 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.121460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cd099-c67b-4b82-a36d-5e29561a30a0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.135404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:33.154426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53c187-41ed-4d9a-829c-bedde9fef698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.155034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53c187-41ed-4d9a-829c-bedde9fef698 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.155314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53c187-41ed-4d9a-829c-bedde9fef698 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.155581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53c187-41ed-4d9a-829c-bedde9fef698 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.156252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53c187-41ed-4d9a-829c-bedde9fef698 None admin] RBAC: Adding query filter params (name=network) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:33.156353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e5f19-b603-431f-b6ae-2cc0bfd1a9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.156551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53c187-41ed-4d9a-829c-bedde9fef698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.157041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e5f19-b603-431f-b6ae-2cc0bfd1a9ed None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.157902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e5f19-b603-431f-b6ae-2cc0bfd1a9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.158004 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53c187-41ed-4d9a-829c-bedde9fef698 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.158185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e5f19-b603-431f-b6ae-2cc0bfd1a9ed None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/25dd5ceedc3040fdb75d9d24a78956a0/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.160392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e5f19-b603-431f-b6ae-2cc0bfd1a9ed None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=25dd5ceedc3040fdb75d9d24a78956a0, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.161992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e5f19-b603-431f-b6ae-2cc0bfd1a9ed None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.162352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 391/809] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:33.163577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88eb5-5f10-4a81-af8c-5f925e1d8efb None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.163865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88eb5-5f10-4a81-af8c-5f925e1d8efb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.164141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88eb5-5f10-4a81-af8c-5f925e1d8efb None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.165538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 392/810] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:33.166029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc88eb5-5f10-4a81-af8c-5f925e1d8efb None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.178796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 419/811] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:33 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/25dd5ceedc3040fdb75d9d24a78956a0/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:33.179611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e5f19-b603-431f-b6ae-2cc0bfd1a9ed None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.290453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:33.290916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc88eb5-5f10-4a81-af8c-5f925e1d8efb None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.319399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e5f19-b603-431f-b6ae-2cc0bfd1a9ed None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.336373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.337122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.337495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.337835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.352769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.354261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.364307 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10cbf8-ce25-4605-90c5-8f12f0d1aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.364956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10cbf8-ce25-4605-90c5-8f12f0d1aa75 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.365230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10cbf8-ce25-4605-90c5-8f12f0d1aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.365533 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10cbf8-ce25-4605-90c5-8f12f0d1aa75 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.366195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10cbf8-ce25-4605-90c5-8f12f0d1aa75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:33.366497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10cbf8-ce25-4605-90c5-8f12f0d1aa75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.367845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10cbf8-ce25-4605-90c5-8f12f0d1aa75 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:33.370492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:33.371395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:33.372319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:33.372319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:33.372319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:33.372319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:33.372319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 393/812] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:33.372319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.386076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 420/813] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:33.387158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54e557-b1ef-4568-8f1b-cc74a03a69ee None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.387443 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54e557-b1ef-4568-8f1b-cc74a03a69ee None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.387712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54e557-b1ef-4568-8f1b-cc74a03a69ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.409717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:33.412710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ce54e557-b1ef-4568-8f1b-cc74a03a69ee None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:33.414055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ce54e557-b1ef-4568-8f1b-cc74a03a69ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:33.488405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10e3f-edc3-48f3-a166-cf2b0eda507a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.505318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 421/814] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:33.505318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54e557-b1ef-4568-8f1b-cc74a03a69ee None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.537857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.538482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.538762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.539037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.539713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] RBAC: Adding query filter params (type=network) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:33.540014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.541444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.546265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 394/815] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:33.547035 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.583025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:33.621988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54e557-b1ef-4568-8f1b-cc74a03a69ee None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.649701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:33.656172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a8bb-e28b-4e48-90bb-80fe8f5ff480 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.659651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.660353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.660710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.661056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.661859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:33.662232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.663700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.676298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 422/816] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:33.677127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.698392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.699536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.699823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.700382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.708531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.710011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:33.721926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:33.723037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 395/817] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:33.724181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.800608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d7076-8b60-4fc7-ba72-644c3ff37b14 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.826986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:33.833741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd8c72-7af1-4461-a230-c47829eda1f9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:33.839196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.839915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.840266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.841013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.841750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.843526 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:33.853662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:33.854836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 06:24:33.855900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:33.857150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 423/818] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:33.858012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.875553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573fd4-91c3-4d97-b38e-dcb93f35ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:33.876713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573fd4-91c3-4d97-b38e-dcb93f35ee4b None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:33.876994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573fd4-91c3-4d97-b38e-dcb93f35ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:33.877315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573fd4-91c3-4d97-b38e-dcb93f35ee4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:33.878014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff573fd4-91c3-4d97-b38e-dcb93f35ee4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:33.879380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff573fd4-91c3-4d97-b38e-dcb93f35ee4b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:33.904093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 396/819] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:24:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:33.904931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573fd4-91c3-4d97-b38e-dcb93f35ee4b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:33.924621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:34.000858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e641ac-f7ff-478d-b589-380c4af58caf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:34.058737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:34.059431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:34.059783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:34.060054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573fd4-91c3-4d97-b38e-dcb93f35ee4b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:34.060179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:34.070886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:34.072462 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:34.087058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:34.088054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:24:34.089052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:34.093380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 424/820] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:34.094203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:34.118927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:34.119569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:34.119855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:34.120133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:34.120899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:34.121205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:34.123647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:34.143979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 397/821] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:34.144849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:34.160585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:34.223749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a260-18b1-487b-9022-54d8c2ebddd1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:34.233192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:34.263651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0176040-3c40-418b-b812-769665539db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:34.264382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0176040-3c40-418b-b812-769665539db3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:34.264664 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0176040-3c40-418b-b812-769665539db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:34.264936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0176040-3c40-418b-b812-769665539db3 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:34.265783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0176040-3c40-418b-b812-769665539db3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:34.266030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de062fe0-12df-45e4-af1e-083aa0a21436 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:34.266147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0176040-3c40-418b-b812-769665539db3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:34.267789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0176040-3c40-418b-b812-769665539db3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:34.273036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 425/822] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jul 28 06:24:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:34.273737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0176040-3c40-418b-b812-769665539db3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:34.309808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:34.310342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:34.310673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:34.310888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:34.312017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:34.312017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:34.313908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:34.331856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 398/823] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:24:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:34.332501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:34.395934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0176040-3c40-418b-b812-769665539db3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:34.408975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:34.440485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398392d-6d13-4273-a982-82a84b75e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:34.441165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398392d-6d13-4273-a982-82a84b75e129 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:34.441483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398392d-6d13-4273-a982-82a84b75e129 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:34.441773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398392d-6d13-4273-a982-82a84b75e129 None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/528c54678648461397f490fbbb698601/roles/e6fe0c4d4ed1454f935b5c9001151260` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:34.444398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84088386-34e0-404f-a998-6fb523f93cca None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:34.459681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398392d-6d13-4273-a982-82a84b75e129 None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=528c54678648461397f490fbbb698601, role_id=e6fe0c4d4ed1454f935b5c9001151260)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:34.461118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398392d-6d13-4273-a982-82a84b75e129 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:34.476331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 426/824] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:34 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/528c54678648461397f490fbbb698601/roles/e6fe0c4d4ed1454f935b5c9001151260 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:34.477622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398392d-6d13-4273-a982-82a84b75e129 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:34.478673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:34.487383 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:34.488359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:34.488840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:34.489262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:34.490391 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:34.490890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:34.492578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:34.499148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 399/825] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Jul 28 06:24:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:34.506367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:34.630590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398392d-6d13-4273-a982-82a84b75e129 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:34.651770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921baf8-25c0-4d8d-919d-99aa2a6ffe6f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:34.678441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd70320-e9ed-435c-86d4-06652d59ade8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:34.679647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd70320-e9ed-435c-86d4-06652d59ade8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:34.680087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd70320-e9ed-435c-86d4-06652d59ade8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:34.680361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd70320-e9ed-435c-86d4-06652d59ade8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:34.695395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8fe03-64e3-46a6-ae9c-f55366734460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:34.696021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8fe03-64e3-46a6-ae9c-f55366734460 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:34.696306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8fe03-64e3-46a6-ae9c-f55366734460 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:34.696579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8fe03-64e3-46a6-ae9c-f55366734460 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:34.697703 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8fe03-64e3-46a6-ae9c-f55366734460 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:34.699630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8fe03-64e3-46a6-ae9c-f55366734460 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:34.718795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:34.728756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 400/826] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:24:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:24:34.772418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd70320-e9ed-435c-86d4-06652d59ade8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:34.774207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd70320-e9ed-435c-86d4-06652d59ade8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:34.897578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 427/827] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:24:34.899457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8fe03-64e3-46a6-ae9c-f55366734460 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:34.954747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:35.015108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8fe03-64e3-46a6-ae9c-f55366734460 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:35.050832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87553cbe-282b-47e6-a5bf-6833de6940d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:35.051535 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87553cbe-282b-47e6-a5bf-6833de6940d2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.051815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87553cbe-282b-47e6-a5bf-6833de6940d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.052089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87553cbe-282b-47e6-a5bf-6833de6940d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.064155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87553cbe-282b-47e6-a5bf-6833de6940d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:35.065935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87553cbe-282b-47e6-a5bf-6833de6940d2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-87553cbe-282b-47e6-a5bf-6833de6940d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:35.083121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:35.084094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:35.085434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:35.085434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:35.085434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:35.085434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:35.085434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 401/828] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:35.090212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd70320-e9ed-435c-86d4-06652d59ade8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:35.152892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:35.197681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd70320-e9ed-435c-86d4-06652d59ade8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:35.208671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3c90a-2d44-4bef-8392-bafd1b6df1c8 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.208986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3c90a-2d44-4bef-8392-bafd1b6df1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.209224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3c90a-2d44-4bef-8392-bafd1b6df1c8 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.210709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 402/829] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:24:35.215683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279d444-7bbc-47eb-a14f-c25add793c55 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.215966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279d444-7bbc-47eb-a14f-c25add793c55 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.216237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279d444-7bbc-47eb-a14f-c25add793c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.235306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b279d444-7bbc-47eb-a14f-c25add793c55 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:35.237206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b279d444-7bbc-47eb-a14f-c25add793c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:35.248213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:35.249199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.249720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.250193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.251776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:35.252292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:35.253924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:35.267033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 428/830] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:35.272390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:35.311156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 403/831] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:35.334563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279d444-7bbc-47eb-a14f-c25add793c55 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:35.362171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:35.397199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7011de-b3f0-4a4d-9a72-ca1ad3718505 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:35.424760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:35.441320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:35.442061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.442744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.443137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.451382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:35.453081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:35.465466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:35.466659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:35.467784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:35.467784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:35.467784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:35.467784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:35.467784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:35.467784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:35.467784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:35.467784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:35.468790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279d444-7bbc-47eb-a14f-c25add793c55 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:35.473600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 429/832] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:35.475902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:35.507076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62acfd-f895-4112-8189-9fedb92fc04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:35.507998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62acfd-f895-4112-8189-9fedb92fc04d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.508431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62acfd-f895-4112-8189-9fedb92fc04d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.508865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62acfd-f895-4112-8189-9fedb92fc04d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.520506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62acfd-f895-4112-8189-9fedb92fc04d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5e62acfd-f895-4112-8189-9fedb92fc04d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:35.526017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:35.527145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:24:35.528071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:35.530737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 404/833] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:35.536850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62acfd-f895-4112-8189-9fedb92fc04d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:35.597024 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cab6e-5174-4b5d-841c-db69c66dbf3c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:35.642703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:35.643661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.644128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.644602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.645566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:35.646065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:35.646173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:35.647801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:35.660161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62acfd-f895-4112-8189-9fedb92fc04d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:35.662348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 430/834] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:35.668083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:35.711786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:35.712324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.712609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.712886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.713702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:35.714009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:35.716677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:35.725257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:35.727498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 405/835] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:35.733284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:35.803459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b70ffe-4d11-4943-92df-b6794ab78826 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:35.845857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:35.846697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.847123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.847475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.848370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=9f558687eefe4a26beb7d69151221660) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:35.848768 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:35.850405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:35.851500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c40e2-722e-45a3-9fa3-64cbf6383503 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:35.863926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 431/836] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:35 2022] GET /identity/v3/role_assignments?user.id=9f558687eefe4a26beb7d69151221660&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:35.895613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:35.896280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:35.896566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:35.896842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:35.909515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:35.911024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:35.927196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:35.928170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:35.929058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:35.929058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:35.929058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:35.929058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:35.929058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 406/837] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:35.934449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:36.049693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba879e-04ce-41c7-8b31-68003bc31c55 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:36.053337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:36.089916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:36.090534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:36.090810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:36.091128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:36.091822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:36.092122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:36.093491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:36.106792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 432/838] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:36.112820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:36.116755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:36.229861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494917d1-c299-43cb-81ae-d33dac5489de None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:36.266594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:36.267246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:36.267531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:36.267808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:36.275942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:36.277530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:36.289715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:36.290691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:36.291630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 407/839] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:24:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:36.299104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:36.405431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf3303-d7dc-4387-a95a-fec44385751d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:36.444798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e9256-81ae-4573-b4dc-7eae428008ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:36.445461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9256-81ae-4573-b4dc-7eae428008ea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:36.445748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9256-81ae-4573-b4dc-7eae428008ea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:36.446020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e9256-81ae-4573-b4dc-7eae428008ea None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:36.446816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e9256-81ae-4573-b4dc-7eae428008ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:36.447127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e9256-81ae-4573-b4dc-7eae428008ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:36.448803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e9256-81ae-4573-b4dc-7eae428008ea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:36.468034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 433/840] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Jul 28 06:24:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:36.474057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:36.549669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:36.581874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce50b09-47c1-4b9f-900a-ad10518af54f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:36.617798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca530544-6d43-4b6c-ad89-cbfb99875c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:36.618424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca530544-6d43-4b6c-ad89-cbfb99875c26 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:36.618707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca530544-6d43-4b6c-ad89-cbfb99875c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:36.618979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca530544-6d43-4b6c-ad89-cbfb99875c26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:36.645392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71988e-3113-4e56-a287-d4e4ea27ab61 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:36.645717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71988e-3113-4e56-a287-d4e4ea27ab61 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:36.645958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71988e-3113-4e56-a287-d4e4ea27ab61 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:36.647530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 434/841] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:36.652404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ea77-0572-493a-b1da-5fd1201cfc22 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:36.652693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ea77-0572-493a-b1da-5fd1201cfc22 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:36.652974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ea77-0572-493a-b1da-5fd1201cfc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:36.673816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b2c0ea77-0572-493a-b1da-5fd1201cfc22 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:36.674947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b2c0ea77-0572-493a-b1da-5fd1201cfc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:36.692652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca530544-6d43-4b6c-ad89-cbfb99875c26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:36.694054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca530544-6d43-4b6c-ad89-cbfb99875c26 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:36.694373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:36.760879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:36.773453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 435/842] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:36.796421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0ea77-0572-493a-b1da-5fd1201cfc22 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:36.816359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 408/843] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:24:36.822616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca530544-6d43-4b6c-ad89-cbfb99875c26 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:36.910056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0ea77-0572-493a-b1da-5fd1201cfc22 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:36.930259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca530544-6d43-4b6c-ad89-cbfb99875c26 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:36.949694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadf1e1-b4f0-4c26-988f-db3769d89607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:36.950730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadf1e1-b4f0-4c26-988f-db3769d89607 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:36.951223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadf1e1-b4f0-4c26-988f-db3769d89607 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:36.951720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadf1e1-b4f0-4c26-988f-db3769d89607 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:36.963496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadf1e1-b4f0-4c26-988f-db3769d89607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:36.967090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:36.967844 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:36.968227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:36.968600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0eadf1e1-b4f0-4c26-988f-db3769d89607 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:36.969408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:36.970439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:24:36.971341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:36.974313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 436/844] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:36.985521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadf1e1-b4f0-4c26-988f-db3769d89607 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:36.985692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:36.985692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:36.998369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:36.999288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:37.000204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:37.000204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:37.000204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:37.000204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:37.001637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:37.005579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 409/845] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:37.006551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.023585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:37.094986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadf1e1-b4f0-4c26-988f-db3769d89607 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.116667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7ec0c-3edc-4eca-9940-fae228750069 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.133894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bcfbb-3ba6-4674-a4c6-6aa5a9d85519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.134526 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bcfbb-3ba6-4674-a4c6-6aa5a9d85519 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.134805 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bcfbb-3ba6-4674-a4c6-6aa5a9d85519 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.135077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bcfbb-3ba6-4674-a4c6-6aa5a9d85519 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.135792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bcfbb-3ba6-4674-a4c6-6aa5a9d85519 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:37.136120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bcfbb-3ba6-4674-a4c6-6aa5a9d85519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.137498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bcfbb-3ba6-4674-a4c6-6aa5a9d85519 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.150826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 437/846] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:37.150826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4501f38-f3c7-4227-9ddc-18867e7412d8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.150826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4501f38-f3c7-4227-9ddc-18867e7412d8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.150826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4501f38-f3c7-4227-9ddc-18867e7412d8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.150826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 438/847] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:37.151699 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4501f38-f3c7-4227-9ddc-18867e7412d8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.161311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf4fe1-75cd-4b8a-a191-7b00f6a08787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.161724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf4fe1-75cd-4b8a-a191-7b00f6a08787 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.162005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf4fe1-75cd-4b8a-a191-7b00f6a08787 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.162281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf4fe1-75cd-4b8a-a191-7b00f6a08787 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.162968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf4fe1-75cd-4b8a-a191-7b00f6a08787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:37.163548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf4fe1-75cd-4b8a-a191-7b00f6a08787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.164641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf4fe1-75cd-4b8a-a191-7b00f6a08787 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.179206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 410/848] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:37.184867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4833ccb-dfd7-49a5-82b7-1c3ba4717d1d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.185087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4833ccb-dfd7-49a5-82b7-1c3ba4717d1d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.185409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4833ccb-dfd7-49a5-82b7-1c3ba4717d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.202839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e4833ccb-dfd7-49a5-82b7-1c3ba4717d1d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:37.204548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e4833ccb-dfd7-49a5-82b7-1c3ba4717d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:37.233362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:37.262559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4501f38-f3c7-4227-9ddc-18867e7412d8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.276976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 411/849] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:37.277652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4833ccb-dfd7-49a5-82b7-1c3ba4717d1d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.282393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:37.311639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.312709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.312990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.313898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.325169 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.326650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:37.341619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:37.342543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:37.343450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:37.343450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:37.343450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:37.343450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:37.343450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 439/850] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:37.343728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.393647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4833ccb-dfd7-49a5-82b7-1c3ba4717d1d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.429721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.430359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.430637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.430906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.440689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.442096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:37.455568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:37.456647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 412/851] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:37.457928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.463567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d10c4-819d-4ed2-a08d-dea7351b8953 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.491834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:37.502050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.502695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.502974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.503256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.503883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.505239 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:37.514406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:37.515594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:37.516924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:37.520474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 440/852] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:37.521357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.537417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:37.581141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982fa84-5f0c-482a-b076-fe7cc386b832 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.615221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.615841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.616119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.616389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.617076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:37.617405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.619191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.631004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e315-d858-4d09-a832-df3eddd2453b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.636171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 413/853] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:37.636171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.670710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.671038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.671318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.671588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.672324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:37.673179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.674643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.692431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 441/854] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:37.693244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.739797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df326df-8bd8-490c-88ac-8a51eb2a82b1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.748377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:37.766165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:37.777651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.778276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.778558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.778833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.779518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:37.779823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.781183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.786415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 414/855] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:37.787201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.811523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c279-1430-4408-8b14-8dfa3ddf583d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.847073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.847846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.848200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.848536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.858468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.859886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:37.871442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:37.872426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 442/856] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:37.873986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.905977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bca5f-f1e9-4838-9f95-6d9639c66c68 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:37.943738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:37.944339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:37.944621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:37.944824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:37.945646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=25dd5ceedc3040fdb75d9d24a78956a0) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:37.945957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:37.947386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:37.961348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 415/857] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/role_assignments?user.id=25dd5ceedc3040fdb75d9d24a78956a0&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:37.961348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:37.995076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd4b25-81a9-4791-9add-902fa8547f7b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:38.010849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:38.034256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:38.045608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218e731-b329-412c-af6b-d534e8976341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:38.046240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218e731-b329-412c-af6b-d534e8976341 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.046530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218e731-b329-412c-af6b-d534e8976341 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.046806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218e731-b329-412c-af6b-d534e8976341 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.047501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218e731-b329-412c-af6b-d534e8976341 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:38.048363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218e731-b329-412c-af6b-d534e8976341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:38.049814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218e731-b329-412c-af6b-d534e8976341 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:38.061602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 443/858] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:38.061602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accd682-8259-45a0-935e-555ec0378024 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.061602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accd682-8259-45a0-935e-555ec0378024 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.061602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accd682-8259-45a0-935e-555ec0378024 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.062736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 444/859] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:38.064938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4ee7-56b3-463d-b748-4a6a6852b3ef None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.065233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4ee7-56b3-463d-b748-4a6a6852b3ef None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.065545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4ee7-56b3-463d-b748-4a6a6852b3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.088592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8c5c4ee7-56b3-463d-b748-4a6a6852b3ef None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:38.089960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8c5c4ee7-56b3-463d-b748-4a6a6852b3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:38.100742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b74fbc-12ae-4fd2-b9a1-4d89f09aba33 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:38.151270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:38.151913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.152192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.152466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.153272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:38.153609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:38.155082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:38.179969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 416/860] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jul 28 06:24:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:38.185778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:38.187125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 445/861] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:38.214490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c4ee7-56b3-463d-b748-4a6a6852b3ef None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:38.312435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7703d-d426-4616-a997-52f15406d007 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:38.328486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:38.342046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c4ee7-56b3-463d-b748-4a6a6852b3ef None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:38.346034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:38.365101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f3dd5-4331-4e45-a8d0-08ad4d85b771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:38.366120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f3dd5-4331-4e45-a8d0-08ad4d85b771 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.366605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f3dd5-4331-4e45-a8d0-08ad4d85b771 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.367598 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f3dd5-4331-4e45-a8d0-08ad4d85b771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.383102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:38.383609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.383890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.384166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.384871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:38.386292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:38.397424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:38.398506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:38.399372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:38.403854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 446/862] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Jul 28 06:24:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:38.404996 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:38.465220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f3dd5-4331-4e45-a8d0-08ad4d85b771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:38.466810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f3dd5-4331-4e45-a8d0-08ad4d85b771 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:38.541507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa336-b448-40e3-afc8-e3b54ff94a3c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:38.574355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 417/863] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 06:24:38.580424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f3dd5-4331-4e45-a8d0-08ad4d85b771 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:38.583112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:38.583902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.584316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.584720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.585581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:38.586035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:38.587542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:38.592906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 447/864] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:38.593484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:38.600286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:38.633648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:38.704312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f3dd5-4331-4e45-a8d0-08ad4d85b771 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:38.720233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1dca0-08c9-492c-bbd9-6da8cb886e17 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:38.752416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:38.753685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.754622 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.755357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.759135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:38.759755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.760038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.760314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.760990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:38.761317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:38.762701 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:38.769155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:38.770909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:38.773494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 448/865] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:38.795127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:38.796123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:38.797021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:38.797021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:38.797021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:38.797021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:38.800333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 418/866] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:38.806333 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:38.915219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:38.922550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c19e01-1862-4699-acbf-26ea568689ef None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:38.960037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:38.961028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:38.961492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:38.961929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:38.962760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:38.963210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:38.964711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:38.978435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 449/867] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:38.984188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:39.050743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:39.092098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15c65-23ae-44b7-a992-bfb5a36c3aa0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:39.128850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:39.129884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:39.130320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:39.130784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:39.138422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:39.140044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:39.152796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:39.153952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:39.155077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:39.155077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:39.155077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:39.155077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:39.155077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:39.155077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:39.155077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:39.155077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:39.161467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 419/868] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:39.163649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:39.276128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06295d-65d5-41da-a56d-ba8620edb1a1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:39.313263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88ba02-6473-4267-b432-6a10cd646997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:39.314619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88ba02-6473-4267-b432-6a10cd646997 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:39.314998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88ba02-6473-4267-b432-6a10cd646997 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:39.315375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88ba02-6473-4267-b432-6a10cd646997 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:39.316225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88ba02-6473-4267-b432-6a10cd646997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:39.316653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88ba02-6473-4267-b432-6a10cd646997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:39.318200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88ba02-6473-4267-b432-6a10cd646997 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:39.331757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 450/869] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:39.337497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:39.442000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02173a63-7055-4177-bd18-00e4e6c72e3d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:39.465477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:39.477742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1657c5-25a7-4822-96d5-9bb2fcef0d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:39.478361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1657c5-25a7-4822-96d5-9bb2fcef0d3d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:39.478643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1657c5-25a7-4822-96d5-9bb2fcef0d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:39.478913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1657c5-25a7-4822-96d5-9bb2fcef0d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:39.479701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1657c5-25a7-4822-96d5-9bb2fcef0d3d None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=528c54678648461397f490fbbb698601) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:39.480009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1657c5-25a7-4822-96d5-9bb2fcef0d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:39.481463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1657c5-25a7-4822-96d5-9bb2fcef0d3d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:39.487926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 420/870] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:39 2022] GET /identity/v3/role_assignments?user.id=528c54678648461397f490fbbb698601&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:40.070521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176393c-5db7-4c22-b095-a1757004ee0b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.071058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176393c-5db7-4c22-b095-a1757004ee0b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.071058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176393c-5db7-4c22-b095-a1757004ee0b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.072635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 451/871] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:40.078231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d5286-3f8a-4c0f-9f24-9e2474071db2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.079054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d5286-3f8a-4c0f-9f24-9e2474071db2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.079533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d5286-3f8a-4c0f-9f24-9e2474071db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.098398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-876d5286-3f8a-4c0f-9f24-9e2474071db2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:40.099978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-876d5286-3f8a-4c0f-9f24-9e2474071db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:40.170874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 421/872] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:40.193121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176393c-5db7-4c22-b095-a1757004ee0b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:40.202259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88201c5-7ead-4a93-8ee9-4dd997972735 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.202548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88201c5-7ead-4a93-8ee9-4dd997972735 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.202823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88201c5-7ead-4a93-8ee9-4dd997972735 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.205903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 422/873] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:24:40.211243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a986c-b495-4d0a-ab65-4774c5c54ddd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.211525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a986c-b495-4d0a-ab65-4774c5c54ddd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.211801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a986c-b495-4d0a-ab65-4774c5c54ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.222710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:40.233857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-886a986c-b495-4d0a-ab65-4774c5c54ddd None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:40.235835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-886a986c-b495-4d0a-ab65-4774c5c54ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:40.315920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 423/874] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:40.322062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176393c-5db7-4c22-b095-a1757004ee0b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:40.341671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a986c-b495-4d0a-ab65-4774c5c54ddd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:40.361229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:40.361906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.362191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.362469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.363100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:40.364469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:40.373733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:40.374764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:40.376274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 452/875] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:40.385468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:40.399170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:40.487514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:40.490589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a986c-b495-4d0a-ab65-4774c5c54ddd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:40.510703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0a3dd-d85c-4b20-8fb0-ebf92ac223c6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:40.530926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5114-e473-4496-a971-72bb3f21ffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:40.531550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5114-e473-4496-a971-72bb3f21ffff None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.532375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5114-e473-4496-a971-72bb3f21ffff None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.532689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5114-e473-4496-a971-72bb3f21ffff None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.533437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c5114-e473-4496-a971-72bb3f21ffff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:40.535301 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c5114-e473-4496-a971-72bb3f21ffff None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:40.551775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a75278-aecc-43aa-badc-3211d7726db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:40.552631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75278-aecc-43aa-badc-3211d7726db4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.553095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75278-aecc-43aa-badc-3211d7726db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.553566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75278-aecc-43aa-badc-3211d7726db4 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.554403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75278-aecc-43aa-badc-3211d7726db4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:40.554860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75278-aecc-43aa-badc-3211d7726db4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:40.556400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75278-aecc-43aa-badc-3211d7726db4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:40.561404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 424/876] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:24:40 2022] POST /identity/v3/services => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:40.566921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 453/877] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:40.574214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5114-e473-4496-a971-72bb3f21ffff None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:40.697152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5114-e473-4496-a971-72bb3f21ffff None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:40.699602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:40.741588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a33b68-047f-4400-966d-7575ef9c6c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:40.742529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a33b68-047f-4400-966d-7575ef9c6c79 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.742969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a33b68-047f-4400-966d-7575ef9c6c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.743451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a33b68-047f-4400-966d-7575ef9c6c79 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.744297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a33b68-047f-4400-966d-7575ef9c6c79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:40.744756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a33b68-047f-4400-966d-7575ef9c6c79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:40.746327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a33b68-047f-4400-966d-7575ef9c6c79 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:40.754886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 425/878] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:40.926833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e982a-ec10-4c48-af4f-ad4c63932233 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.927495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e982a-ec10-4c48-af4f-ad4c63932233 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.927997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e982a-ec10-4c48-af4f-ad4c63932233 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.929857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 454/879] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:40.935288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0687991-49e1-49b0-86d6-11d4993d2919 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:40.935833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0687991-49e1-49b0-86d6-11d4993d2919 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:40.936340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0687991-49e1-49b0-86d6-11d4993d2919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:40.955966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c0687991-49e1-49b0-86d6-11d4993d2919 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:40.957506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c0687991-49e1-49b0-86d6-11d4993d2919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:41.028881 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 426/880] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:41.052223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e982a-ec10-4c48-af4f-ad4c63932233 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:41.164612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e982a-ec10-4c48-af4f-ad4c63932233 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:41.203962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee4024-4348-4678-b4a0-ed7351403f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:41.204762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee4024-4348-4678-b4a0-ed7351403f05 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:41.205043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee4024-4348-4678-b4a0-ed7351403f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:41.205338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee4024-4348-4678-b4a0-ed7351403f05 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:41.206046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee4024-4348-4678-b4a0-ed7351403f05 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:41.207401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee4024-4348-4678-b4a0-ed7351403f05 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:41.227128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 455/881] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:24:41 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:41.623531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4eb7d-0208-498b-9fb8-b40d43353728 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:41.624555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4eb7d-0208-498b-9fb8-b40d43353728 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:41.625052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4eb7d-0208-498b-9fb8-b40d43353728 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:41.627175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 427/882] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:41.632149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db533daf-8220-4f26-b9bd-7c6568a407e7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:41.632442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db533daf-8220-4f26-b9bd-7c6568a407e7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:41.632720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db533daf-8220-4f26-b9bd-7c6568a407e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:41.640787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:41.652949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-db533daf-8220-4f26-b9bd-7c6568a407e7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:41.654222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-db533daf-8220-4f26-b9bd-7c6568a407e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:41.730614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 456/883] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:41.754850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4eb7d-0208-498b-9fb8-b40d43353728 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:41.855733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:41.862985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4eb7d-0208-498b-9fb8-b40d43353728 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:41.898238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8c01f-c91e-433d-8155-915906753591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:41.898803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8c01f-c91e-433d-8155-915906753591 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:41.899080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8c01f-c91e-433d-8155-915906753591 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:41.899352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8c01f-c91e-433d-8155-915906753591 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:41.910338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8c01f-c91e-433d-8155-915906753591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c1d8c01f-c91e-433d-8155-915906753591 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:41.915312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:41.916268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:41.917176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 428/884] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jul 28 06:24:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:41.927127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db533daf-8220-4f26-b9bd-7c6568a407e7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:42.040367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db533daf-8220-4f26-b9bd-7c6568a407e7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:42.070846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:42.078105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:42.078803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.079142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.079420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.080193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:42.080503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:42.081945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:42.091077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 457/885] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jul 28 06:24:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:42.097512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8c01f-c91e-433d-8155-915906753591 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:42.200391 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8c01f-c91e-433d-8155-915906753591 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:42.233648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:42.234286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.234565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.235351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.245896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:42.247340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:42.262874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:42.263807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:42.264730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:42.264730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:42.264730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:42.264730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:42.264730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 429/886] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:42.270043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:42.382633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003810d-fe4b-4c55-9955-82112c7aea46 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:42.419378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:42.420280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.420723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.421169 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.422033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:42.422520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:42.424144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:42.443417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 458/887] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:42.449770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:42.461484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:42.563144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31298f5a-91c5-419d-b367-258b36f3fa5f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:42.598704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0efe9-f034-4270-97a5-ca5a2120d4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:42.599340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0efe9-f034-4270-97a5-ca5a2120d4ce None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.599617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0efe9-f034-4270-97a5-ca5a2120d4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.599889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0efe9-f034-4270-97a5-ca5a2120d4ce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.607643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0efe9-f034-4270-97a5-ca5a2120d4ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:42.609081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0efe9-f034-4270-97a5-ca5a2120d4ce None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-06d0efe9-f034-4270-97a5-ca5a2120d4ce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:42.621374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:42.622348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:42.623307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 430/888] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:24:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:42.630864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:42.671127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64e046-52da-4edb-be40-424a41303213 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.671414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64e046-52da-4edb-be40-424a41303213 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.671896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64e046-52da-4edb-be40-424a41303213 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.674227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 431/889] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:24:42.679197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be87c3-f134-46b2-9d4a-653235dbdf77 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.679658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be87c3-f134-46b2-9d4a-653235dbdf77 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.679939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be87c3-f134-46b2-9d4a-653235dbdf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.698036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:42.698964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-12be87c3-f134-46b2-9d4a-653235dbdf77 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:42.700315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-12be87c3-f134-46b2-9d4a-653235dbdf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:42.744808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e997684-f1d4-49e4-b697-b51cdbdc315d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:42.772547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:42.778387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 432/890] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:42.788684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:42.789527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.789951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.790368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.791298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:42.791803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:42.793375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:42.800808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be87c3-f134-46b2-9d4a-653235dbdf77 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:42.815260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 459/891] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jul 28 06:24:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:42.822442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:42.921756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be87c3-f134-46b2-9d4a-653235dbdf77 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:42.951856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ac89-f9bc-47a2-9b71-663e1c19078c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:42.964609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:42.971232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578c648-922a-4dcc-a531-a08590b14048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:42.972184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c648-922a-4dcc-a531-a08590b14048 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.972474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c648-922a-4dcc-a531-a08590b14048 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.972748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578c648-922a-4dcc-a531-a08590b14048 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.973827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578c648-922a-4dcc-a531-a08590b14048 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:42.975206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578c648-922a-4dcc-a531-a08590b14048 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-d578c648-922a-4dcc-a531-a08590b14048 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:42.986290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:42.987344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:42.988600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 433/892] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:42.989433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ae93-8efa-4963-b7d4-d73a56a53790 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.989433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ae93-8efa-4963-b7d4-d73a56a53790 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.989714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ae93-8efa-4963-b7d4-d73a56a53790 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:42.991069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 434/893] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:42.995553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16ae93-8efa-4963-b7d4-d73a56a53790 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:42.997382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c4639-518a-4425-97f0-d74cb0af1e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:42.998246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c4639-518a-4425-97f0-d74cb0af1e4d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:42.998543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c4639-518a-4425-97f0-d74cb0af1e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:42.998827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c4639-518a-4425-97f0-d74cb0af1e4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.059426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:43.088894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c4639-518a-4425-97f0-d74cb0af1e4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.090966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c4639-518a-4425-97f0-d74cb0af1e4d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.104553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16ae93-8efa-4963-b7d4-d73a56a53790 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:43.141991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282d951-7d06-4b68-83dc-f6ee42161639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:43.142636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282d951-7d06-4b68-83dc-f6ee42161639 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.142879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282d951-7d06-4b68-83dc-f6ee42161639 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.143148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282d951-7d06-4b68-83dc-f6ee42161639 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.143818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282d951-7d06-4b68-83dc-f6ee42161639 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:43.144118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282d951-7d06-4b68-83dc-f6ee42161639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.145488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282d951-7d06-4b68-83dc-f6ee42161639 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.150444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 435/894] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:43.151599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bc38d-0a2e-44f1-aead-7713902d0739 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.151882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bc38d-0a2e-44f1-aead-7713902d0739 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.152150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bc38d-0a2e-44f1-aead-7713902d0739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.170569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5b6bc38d-0a2e-44f1-aead-7713902d0739 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:43.171729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5b6bc38d-0a2e-44f1-aead-7713902d0739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:43.204430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 460/895] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:24:43.205271 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c4639-518a-4425-97f0-d74cb0af1e4d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.217819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:43.249414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 436/896] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:43.249414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bc38d-0a2e-44f1-aead-7713902d0739 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.319319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c4639-518a-4425-97f0-d74cb0af1e4d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:43.322316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:43.358053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:43.358676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.358958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.359230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.359950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:43.360802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.362419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.367828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 461/897] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:43.368605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.376501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bc38d-0a2e-44f1-aead-7713902d0739 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:43.417209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:43.417887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.418170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.418443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.429530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.430981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:43.447519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:43.448818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:43.449949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:43.449949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:43.449949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:43.449949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:43.454228 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 437/898] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:43.455701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.458859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:43.484739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7840e-d98b-4b5e-b58c-7d65bd485780 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:43.520269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d8de6-0e5e-42a4-a676-67b2829be7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:43.520810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8de6-0e5e-42a4-a676-67b2829be7c2 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.521088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8de6-0e5e-42a4-a676-67b2829be7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.521391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8de6-0e5e-42a4-a676-67b2829be7c2 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.522066 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d8de6-0e5e-42a4-a676-67b2829be7c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.523406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d8de6-0e5e-42a4-a676-67b2829be7c2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.538692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 462/899] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:24:43 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:43.539957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7555ba-4edc-40eb-8ab4-973e4868b13c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.540244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7555ba-4edc-40eb-8ab4-973e4868b13c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.540517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7555ba-4edc-40eb-8ab4-973e4868b13c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.541748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 463/900] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:43.544602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7555ba-4edc-40eb-8ab4-973e4868b13c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.567969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27638562-a10c-4ac4-a42f-45218f5eb411 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:43.580780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:43.617501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b43f4-4a1a-4b61-a40d-c64fd8e6604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:43.618221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b43f4-4a1a-4b61-a40d-c64fd8e6604a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.618503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b43f4-4a1a-4b61-a40d-c64fd8e6604a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.618777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b43f4-4a1a-4b61-a40d-c64fd8e6604a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.619543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b43f4-4a1a-4b61-a40d-c64fd8e6604a None admin] RBAC: Adding query filter params (interface=public, service_id=27200f4f32a944cba007d0c7244ea160, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:43.619857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b43f4-4a1a-4b61-a40d-c64fd8e6604a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.621590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b43f4-4a1a-4b61-a40d-c64fd8e6604a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.633115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 438/901] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jul 28 06:24:43 2022] GET /identity/v3/endpoints?service_id=27200f4f32a944cba007d0c7244ea160&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:43.634720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf74f3d-60e6-4e0a-80fb-d3c018f4f49a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.635015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf74f3d-60e6-4e0a-80fb-d3c018f4f49a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.635298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf74f3d-60e6-4e0a-80fb-d3c018f4f49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.655135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fdf74f3d-60e6-4e0a-80fb-d3c018f4f49a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:43.656366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fdf74f3d-60e6-4e0a-80fb-d3c018f4f49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:43.667912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7555ba-4edc-40eb-8ab4-973e4868b13c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:43.709572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:43.710198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.710476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.710885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.711434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:43.712310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.713332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.714622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:43.728341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 464/902] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:43.729036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.743142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 439/903] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:43.743756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf74f3d-60e6-4e0a-80fb-d3c018f4f49a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.843862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de3f44-8e01-4748-80b6-8fff8300e8ae None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:43.850292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:43.854493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf74f3d-60e6-4e0a-80fb-d3c018f4f49a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:43.878459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c7e87-cbf0-47a8-a4c2-21c24fec3dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:43.879087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c7e87-cbf0-47a8-a4c2-21c24fec3dcb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.879369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c7e87-cbf0-47a8-a4c2-21c24fec3dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.879642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c7e87-cbf0-47a8-a4c2-21c24fec3dcb None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.880302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c7e87-cbf0-47a8-a4c2-21c24fec3dcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.881681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c7e87-cbf0-47a8-a4c2-21c24fec3dcb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.897340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 465/904] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:24:43 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:24:43.897340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c7e87-cbf0-47a8-a4c2-21c24fec3dcb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.902769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:43.903477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:43.903755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:43.904122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:43.913330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:43.914947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:43.929512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:43.930549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:43.931467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 440/905] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:43.940850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:43.976501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:44.031572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c7e87-cbf0-47a8-a4c2-21c24fec3dcb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:44.075931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:44.076671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:44.076952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:44.077229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:44.077872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:44.079643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:44.085923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db3f7f-db88-4b7d-a019-f1eb6601348a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:44.092965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:44.094684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:44.095929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:44.098476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 466/906] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Jul 28 06:24:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:44.105017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:44.130255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:44.131150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:44.131580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:44.132043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:44.132946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:44.133443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:44.135029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:44.148357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 441/907] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:44.154400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:44.179333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:44.231408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe04d3-7999-4d6d-afcc-4fae1e1889c7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:44.259179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:44.273544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:44.274546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:44.275041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:44.275524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:44.276417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:44.276924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:44.278522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:44.284758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 467/908] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:44.289973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:44.291396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9998-1094-4959-9ebf-eb2cf1df4b81 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:44.333785 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:44.334482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:44.334731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:44.335011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:44.335814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] RBAC: Adding query filter params (role.id=030852a56a8145e3b8b7c785b4ace178, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=528c54678648461397f490fbbb698601) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:44.336260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:44.337608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:44.349399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 442/909] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:44 2022] GET /identity/v3/role_assignments?user.id=528c54678648461397f490fbbb698601&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=030852a56a8145e3b8b7c785b4ace178 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:44.418603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83191dc5-5366-4d68-91f9-715fbd773f6b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:44.462300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:44.462937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:44.463223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:44.463499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:44.464190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:44.464495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:44.465931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:44.472276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 468/910] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:44.478483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:44.568572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:44.593448 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb92ad3-7acd-48c3-8bc9-3c4f10488f9a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:44.629856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57751a13-ccab-4e81-be75-2ccb3160634e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:44.630388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57751a13-ccab-4e81-be75-2ccb3160634e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:44.630670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57751a13-ccab-4e81-be75-2ccb3160634e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:44.630947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57751a13-ccab-4e81-be75-2ccb3160634e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:44.631705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57751a13-ccab-4e81-be75-2ccb3160634e None admin] RBAC: Adding query filter params (interface=public, service_id=6a00da58ddc74ea6ab940dd481e4f2e5, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:44.632015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57751a13-ccab-4e81-be75-2ccb3160634e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:44.633458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57751a13-ccab-4e81-be75-2ccb3160634e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:44.639261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 443/911] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jul 28 06:24:44 2022] GET /identity/v3/endpoints?service_id=6a00da58ddc74ea6ab940dd481e4f2e5&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:44.645464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:44.703158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:44.762745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d39d47-caa0-4872-8385-8ff8be491010 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:44.798959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8808a-e535-4899-9641-6ba123581c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:44.799589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8808a-e535-4899-9641-6ba123581c5c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:44.799870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8808a-e535-4899-9641-6ba123581c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:44.800192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8808a-e535-4899-9641-6ba123581c5c None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:44.800857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8808a-e535-4899-9641-6ba123581c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:44.802236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8808a-e535-4899-9641-6ba123581c5c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:44.808727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 469/912] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:24:44 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:24:45.605685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fa185-6f72-45a7-add6-c144585f633b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:45.606062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fa185-6f72-45a7-add6-c144585f633b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:45.606260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fa185-6f72-45a7-add6-c144585f633b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:45.607868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 444/913] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:45.613509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bc2dc-1633-4784-89be-e87f8eda7052 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:45.613991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bc2dc-1633-4784-89be-e87f8eda7052 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:45.614073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bc2dc-1633-4784-89be-e87f8eda7052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:45.632882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c16bc2dc-1633-4784-89be-e87f8eda7052 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:45.634109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c16bc2dc-1633-4784-89be-e87f8eda7052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:45.709519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 470/914] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:45.731168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fa185-6f72-45a7-add6-c144585f633b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:45.839299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fa185-6f72-45a7-add6-c144585f633b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:45.874679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:45.875879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:45.876269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:45.876572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:45.877196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:45.878591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:45.887894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:45.888943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:45.889847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 445/915] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:45.896056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bc2dc-1633-4784-89be-e87f8eda7052 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:45.945982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:46.008686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bc2dc-1633-4784-89be-e87f8eda7052 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:46.046441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c7c80-b316-40d7-9fe8-f0f6e156ae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:46.047072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7c80-b316-40d7-9fe8-f0f6e156ae82 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.047886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7c80-b316-40d7-9fe8-f0f6e156ae82 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.048223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7c80-b316-40d7-9fe8-f0f6e156ae82 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.048906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c7c80-b316-40d7-9fe8-f0f6e156ae82 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:46.049214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c7c80-b316-40d7-9fe8-f0f6e156ae82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:46.050607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c7c80-b316-40d7-9fe8-f0f6e156ae82 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:46.055930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 471/916] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:46.062365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.076845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:46.160901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1490d83-3a86-4daf-8923-722d6c8c8b78 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.161249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1490d83-3a86-4daf-8923-722d6c8c8b78 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.161470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1490d83-3a86-4daf-8923-722d6c8c8b78 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.162976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 472/917] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:24:46.168112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8629-1b00-4b05-bd91-53b41985cd00 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.168623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8629-1b00-4b05-bd91-53b41985cd00 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.169127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8629-1b00-4b05-bd91-53b41985cd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.182553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd3272-a1c9-4718-a6ba-b711efaa409c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:46.189770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-863c8629-1b00-4b05-bd91-53b41985cd00 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:46.191270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-863c8629-1b00-4b05-bd91-53b41985cd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:46.226969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:46.227591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.227874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.228163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.228818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:46.229122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:46.230788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:46.242886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 446/918] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:46.245491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.277501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 473/919] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:46.300916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8629-1b00-4b05-bd91-53b41985cd00 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.336927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:46.344983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:46.368998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc89d9-f323-4ef4-a4db-64659bf0c609 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:46.412666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1939a-d1f0-463b-8ab7-1ba8887d522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:46.413287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1939a-d1f0-463b-8ab7-1ba8887d522e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.413594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1939a-d1f0-463b-8ab7-1ba8887d522e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.413867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1939a-d1f0-463b-8ab7-1ba8887d522e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.414615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1939a-d1f0-463b-8ab7-1ba8887d522e None admin] RBAC: Adding query filter params (interface=public, service_id=182615cceb7949e9a10867ac890f1e0e, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:46.414919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1939a-d1f0-463b-8ab7-1ba8887d522e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:46.415926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8629-1b00-4b05-bd91-53b41985cd00 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:46.416282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1939a-d1f0-463b-8ab7-1ba8887d522e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:46.425914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 447/920] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jul 28 06:24:46 2022] GET /identity/v3/endpoints?service_id=182615cceb7949e9a10867ac890f1e0e&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:46.425914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423796-9ccd-4f42-b5e4-1cda4a4b63ec None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.425914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423796-9ccd-4f42-b5e4-1cda4a4b63ec None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.425914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423796-9ccd-4f42-b5e4-1cda4a4b63ec None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.425914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 448/921] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:46.429983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ccc98-5123-43d1-9418-ce858540be9f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.430271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ccc98-5123-43d1-9418-ce858540be9f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.430814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ccc98-5123-43d1-9418-ce858540be9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.450738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c38ccc98-5123-43d1-9418-ce858540be9f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:46.451945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c38ccc98-5123-43d1-9418-ce858540be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:46.478696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:46.479410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.479697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.479984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.480601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:46.484818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:46.493073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:46.494233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 06:24:46.495117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:46.499460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 474/922] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:46.500903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.531434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 449/923] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:46.532783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ccc98-5123-43d1-9418-ce858540be9f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.604008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:46.623133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5bb29-37b7-4978-b6fb-c6bb27545aa6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:46.624674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:46.643498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ccc98-5123-43d1-9418-ce858540be9f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:46.662415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe46a6-3220-47c6-ae1f-2f2673835f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:46.663037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe46a6-3220-47c6-ae1f-2f2673835f58 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.663322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe46a6-3220-47c6-ae1f-2f2673835f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.663662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe46a6-3220-47c6-ae1f-2f2673835f58 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.664326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe46a6-3220-47c6-ae1f-2f2673835f58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:46.665723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe46a6-3220-47c6-ae1f-2f2673835f58 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:46.671706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 475/924] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:24:46 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:24:46.680730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:46.681838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.682303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.682423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe46a6-3220-47c6-ae1f-2f2673835f58 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.682875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.683765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:46.684262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:46.685898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:46.699053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 450/925] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:46.700330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.820864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a008d-55dd-476a-a700-5afb9724ead0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:46.834627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe46a6-3220-47c6-ae1f-2f2673835f58 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:46.858544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:46.868432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:46.869055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.869367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.869636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.870317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:46.870620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:46.871985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:46.885335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 451/926] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Jul 28 06:24:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:46.886427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.887783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590c6c3-d9cb-44cc-9e7d-212b8caf7974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:46.888496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c6c3-d9cb-44cc-9e7d-212b8caf7974 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.888831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c6c3-d9cb-44cc-9e7d-212b8caf7974 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.889107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c6c3-d9cb-44cc-9e7d-212b8caf7974 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.900085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590c6c3-d9cb-44cc-9e7d-212b8caf7974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3590c6c3-d9cb-44cc-9e7d-212b8caf7974 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:46.905513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:46.906540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:46.907437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 476/927] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jul 28 06:24:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:46.908669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041562-75b9-4777-9eba-5c6eb47207d6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:46.908669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041562-75b9-4777-9eba-5c6eb47207d6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:46.908986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041562-75b9-4777-9eba-5c6eb47207d6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:46.910338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 477/928] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:46.915206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041562-75b9-4777-9eba-5c6eb47207d6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:46.947164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:46.999159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9bf8c-b09d-4dac-a198-c68b861bd6f8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.029640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041562-75b9-4777-9eba-5c6eb47207d6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.037782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243a511-a98c-4c26-9d93-93d1b4e4769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.038410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243a511-a98c-4c26-9d93-93d1b4e4769b None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.038693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243a511-a98c-4c26-9d93-93d1b4e4769b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.038970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243a511-a98c-4c26-9d93-93d1b4e4769b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.039693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243a511-a98c-4c26-9d93-93d1b4e4769b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.041052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243a511-a98c-4c26-9d93-93d1b4e4769b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.073661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 452/929] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:24:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:47.073661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc87a3-ec1a-4153-9ebb-ce619bd65d75 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.073661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc87a3-ec1a-4153-9ebb-ce619bd65d75 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.073661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc87a3-ec1a-4153-9ebb-ce619bd65d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.084360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.085094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.085504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.086433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.086687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:47.087062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.088675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.093468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-50cc87a3-ec1a-4153-9ebb-ce619bd65d75 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:47.095767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-50cc87a3-ec1a-4153-9ebb-ce619bd65d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:47.099274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 478/930] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jul 28 06:24:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:47.105279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.158982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:47.193319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 453/931] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:47.217819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc87a3-ec1a-4153-9ebb-ce619bd65d75 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.237724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:47.258828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f614-2a63-477d-9e79-448d69e3f737 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.303011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.303661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.303948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.304226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.318604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.320603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:47.336325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:47.337247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:47.338206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:47.338206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:47.338206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:47.338206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:47.338206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 479/932] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:47.342651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc87a3-ec1a-4153-9ebb-ce619bd65d75 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.343467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.387641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.389584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.389781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.390034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.390675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.392827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:47.409245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:47.410347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 06:24:47.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:47.418593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 454/933] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Jul 28 06:24:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:47.421345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.472944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c2328-3e57-456e-b271-f89a1b1c0fe3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.487938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:47.513348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.513991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.514275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.514552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.515524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:47.516342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.518415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.522436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:47.537441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 480/934] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:47.545410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.559086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72f88e-3a5d-4075-8a0a-0fc543753b1c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.595652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.596198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.596481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.596759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.597479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:47.597787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.599151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.604331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 455/935] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:47.611708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.663499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c794-270c-41a4-9adc-edd2bf97c521 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.700531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.701176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.701523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.701803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.709730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.711174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.723317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70a68d-d970-4623-ab95-e60f97a667f0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:47.723615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:47.724534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:47.725543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 481/936] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:24:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:47.730408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.734502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:47.762551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.763448 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.763901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.764325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.765154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:47.765699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.767473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.777443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 456/937] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:47.779174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.791896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:47.840213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d90eb5-8de4-4a50-962e-63b578156b72 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.877078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.877656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.877937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.878217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.879015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:47.879323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.880832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.893144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86accd2-febe-4a06-a946-e62a8760d4b3 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:47.905747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 482/938] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jul 28 06:24:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:47.907548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:47.933443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a387c53-d2dc-4a35-9fd0-2f7064277f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:47.934345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a387c53-d2dc-4a35-9fd0-2f7064277f65 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:47.934783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a387c53-d2dc-4a35-9fd0-2f7064277f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:47.935258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a387c53-d2dc-4a35-9fd0-2f7064277f65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:47.936171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a387c53-d2dc-4a35-9fd0-2f7064277f65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:47.937806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a387c53-d2dc-4a35-9fd0-2f7064277f65 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:47.969923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 457/939] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:24:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:48.032832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13fc0a-ecfa-4f67-acb7-5597f6f7a381 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:48.049485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:48.083952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:48.084983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:48.085532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:48.086054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:48.098326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:48.100241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:48.116956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:48.119209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:48.120339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:48.120339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:48.120339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:48.120339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:48.122221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 483/940] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:48.128445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a387c53-d2dc-4a35-9fd0-2f7064277f65 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:48.164121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:48.243061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a387c53-d2dc-4a35-9fd0-2f7064277f65 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:48.277705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:48.278326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:48.278586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:48.278860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:48.279549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:48.279847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:48.281215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:48.294226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 458/941] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:48.300180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:48.410299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8a9bc-ffe8-42be-8ade-b7ec27b4103c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:48.447062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:48.448137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:48.448864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:48.449363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:48.456777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:48.458253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:48.470081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:48.470979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:48.472027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 484/942] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:48.477014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:48.579646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:48.590136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77259a7-c6b8-454f-bfe7-a56900afd09b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:48.627858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707a376-29e2-4c83-ba6d-e96b0aa63e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:48.628476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707a376-29e2-4c83-ba6d-e96b0aa63e13 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:48.628761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707a376-29e2-4c83-ba6d-e96b0aa63e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:48.629027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707a376-29e2-4c83-ba6d-e96b0aa63e13 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:48.629796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707a376-29e2-4c83-ba6d-e96b0aa63e13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:48.630098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707a376-29e2-4c83-ba6d-e96b0aa63e13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:48.631511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707a376-29e2-4c83-ba6d-e96b0aa63e13 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:48.644578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 459/943] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:48.650395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:48.677062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:48.757697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ecdb7-bdef-44c8-b674-62e41d09ec2f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:48.792567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea3342-e08b-4bd1-8efe-4903051e81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:48.793315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3342-e08b-4bd1-8efe-4903051e81f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:48.793601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3342-e08b-4bd1-8efe-4903051e81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:48.793880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3342-e08b-4bd1-8efe-4903051e81f6 None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/528c54678648461397f490fbbb698601/roles/030852a56a8145e3b8b7c785b4ace178` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:48.796154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea3342-e08b-4bd1-8efe-4903051e81f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=528c54678648461397f490fbbb698601, role_id=030852a56a8145e3b8b7c785b4ace178)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:48.798039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea3342-e08b-4bd1-8efe-4903051e81f6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:48.824502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 485/944] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:24:48 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/528c54678648461397f490fbbb698601/roles/030852a56a8145e3b8b7c785b4ace178 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:24:48.868306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be7df6-ead9-4562-a58a-12c72a9afc55 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:48.868953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be7df6-ead9-4562-a58a-12c72a9afc55 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:48.869496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be7df6-ead9-4562-a58a-12c72a9afc55 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:48.871467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 460/945] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:48.878896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e98e84-51c8-43cf-a654-0dfe97e9298e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:48.879566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e98e84-51c8-43cf-a654-0dfe97e9298e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:48.880053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e98e84-51c8-43cf-a654-0dfe97e9298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:48.901976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-95e98e84-51c8-43cf-a654-0dfe97e9298e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:48.903614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-95e98e84-51c8-43cf-a654-0dfe97e9298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:48.984334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 486/946] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:49.012640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be7df6-ead9-4562-a58a-12c72a9afc55 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:49.155095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be7df6-ead9-4562-a58a-12c72a9afc55 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:49.190911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9344d-e8f4-4f98-a460-57d7e62ead41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:49.191562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9344d-e8f4-4f98-a460-57d7e62ead41 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:49.191841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9344d-e8f4-4f98-a460-57d7e62ead41 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:49.192111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9344d-e8f4-4f98-a460-57d7e62ead41 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:49.192736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9344d-e8f4-4f98-a460-57d7e62ead41 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:49.194147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9344d-e8f4-4f98-a460-57d7e62ead41 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-bce9344d-e8f4-4f98-a460-57d7e62ead41 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:49.204153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:49.205182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.206466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 461/947] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:49.215642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e98e84-51c8-43cf-a654-0dfe97e9298e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:49.238430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:49.299642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414f6b6-3151-4b9b-94af-3ae5b0ed5ae4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:49.300251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414f6b6-3151-4b9b-94af-3ae5b0ed5ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:49.301324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414f6b6-3151-4b9b-94af-3ae5b0ed5ae4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:49.303201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 462/948] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:24:49.310959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f6406-ff38-44b9-ae0a-4085a77b3515 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:49.313412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f6406-ff38-44b9-ae0a-4085a77b3515 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:49.313412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f6406-ff38-44b9-ae0a-4085a77b3515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:49.342248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c37f6406-ff38-44b9-ae0a-4085a77b3515 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:49.343545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c37f6406-ff38-44b9-ae0a-4085a77b3515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:49.353714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e98e84-51c8-43cf-a654-0dfe97e9298e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:49.369735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:49.417332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:49.417709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:49.417991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:49.418265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:49.418952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:49.419258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:49.420624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:49.439238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 487/949] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:49.440231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:49.457895 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 463/950] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:49.479910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f6406-ff38-44b9-ae0a-4085a77b3515 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:49.543829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:49.591005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749411b4-8695-4aef-ae72-374f54b7e7e0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:49.662928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:49.663726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:49.664113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:49.664491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:49.665319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:49.665734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:49.667201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:49.673690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f6406-ff38-44b9-ae0a-4085a77b3515 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:49.678282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 488/951] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:49.679676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:49.713120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:49.714007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:49.714434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:49.714847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:49.715667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:49.717251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:49.720200 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING keystone.server.flask.common [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 06:24:49.735100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:49.741024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.sql.core [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 06:24:49.741024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 06:24:49.741024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [parameters: {'id': 'febe9d2662cd44709963748429fb38f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 06:24:49.741024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80276) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id febe9d2662cd44709963748429fb38f8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id febe9d2662cd44709963748429fb38f8. Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:24:49.760183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result.read() Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 06:24:49.761116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application yield resource Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise self.value Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application session.commit() Jul 28 06:24:49.762124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.session.flush() Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application compat.raise_( Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 06:24:49.763030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 06:24:49.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application result.read() Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:24:49.764802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [parameters: {'id': 'febe9d2662cd44709963748429fb38f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:49.765973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:49.766797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id febe9d2662cd44709963748429fb38f8. Jul 28 06:24:49.767753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.771245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 464/952] 10.222.0.20 () {60 vars in 1140 bytes} [Thu Jul 28 06:24:49 2022] POST /identity/v3/projects => generated 262 bytes in 292 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:49.781555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:49.806158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1ac3a-4243-4f5c-8b8e-4d91712404f9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:49.849562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d2310-4452-4a86-a1ea-ca43a56c90d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:49.850215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d2310-4452-4a86-a1ea-ca43a56c90d2 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:49.850499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d2310-4452-4a86-a1ea-ca43a56c90d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:49.850776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d2310-4452-4a86-a1ea-ca43a56c90d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:49.851599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d2310-4452-4a86-a1ea-ca43a56c90d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:49.852947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d2310-4452-4a86-a1ea-ca43a56c90d2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:49.864604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:49.886642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 489/953] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:24:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:49.898636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba42c78-55ff-4e10-8cd6-57c548c2b056 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:49.951324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa3c5d-a496-4c68-9167-11f0f537e262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:49.952327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3c5d-a496-4c68-9167-11f0f537e262 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:49.952840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3c5d-a496-4c68-9167-11f0f537e262 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:49.953347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa3c5d-a496-4c68-9167-11f0f537e262 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:49.961749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa3c5d-a496-4c68-9167-11f0f537e262 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:49.963661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa3c5d-a496-4c68-9167-11f0f537e262 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-8caa3c5d-a496-4c68-9167-11f0f537e262 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:49.976961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:49.978179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:49.979341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:49.979341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:49.979341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:49.979341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:49.979341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:49.979341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:49.979341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:49.979341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:49.981575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 465/954] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:49.988451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d2310-4452-4a86-a1ea-ca43a56c90d2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:50.105824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d2310-4452-4a86-a1ea-ca43a56c90d2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:50.146264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a33247-458a-4eb6-9ac9-1077579e5fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:50.146890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a33247-458a-4eb6-9ac9-1077579e5fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:50.147174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a33247-458a-4eb6-9ac9-1077579e5fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:50.147446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a33247-458a-4eb6-9ac9-1077579e5fa9 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:50.148169 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a33247-458a-4eb6-9ac9-1077579e5fa9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:50.148470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a33247-458a-4eb6-9ac9-1077579e5fa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:50.149928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a33247-458a-4eb6-9ac9-1077579e5fa9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:50.172735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 490/955] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:50.300744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad530b-bbcc-4e8d-8a19-062dff706807 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:50.301270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad530b-bbcc-4e8d-8a19-062dff706807 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:50.301379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad530b-bbcc-4e8d-8a19-062dff706807 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:50.303288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 466/956] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:50.309075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03bed-518a-4c56-bc46-646cd6c13e64 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:50.309384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03bed-518a-4c56-bc46-646cd6c13e64 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:50.309853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03bed-518a-4c56-bc46-646cd6c13e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:50.330357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-15c03bed-518a-4c56-bc46-646cd6c13e64 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:50.331545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-15c03bed-518a-4c56-bc46-646cd6c13e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:50.387324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:50.405637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 491/957] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:50.427765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad530b-bbcc-4e8d-8a19-062dff706807 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:50.491908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:50.538553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad530b-bbcc-4e8d-8a19-062dff706807 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:50.575062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:50.575712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:50.575988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:50.576259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:50.576889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:50.578316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:50.587629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:50.588700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:50.589613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 467/958] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:50.595006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03bed-518a-4c56-bc46-646cd6c13e64 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:50.697535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03bed-518a-4c56-bc46-646cd6c13e64 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:50.732840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb56e4c-eab8-4475-94f4-cbc47fdb7a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:50.733501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb56e4c-eab8-4475-94f4-cbc47fdb7a4c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:50.733791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb56e4c-eab8-4475-94f4-cbc47fdb7a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:50.734043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb56e4c-eab8-4475-94f4-cbc47fdb7a4c None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:50.734919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb56e4c-eab8-4475-94f4-cbc47fdb7a4c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:50.735246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb56e4c-eab8-4475-94f4-cbc47fdb7a4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:50.736662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb56e4c-eab8-4475-94f4-cbc47fdb7a4c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:50.741932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 492/959] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:50.747497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:50.853508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1e14a-4d25-4b57-b158-89a47e2d2f04 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:50.873955 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:50.888709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2b197-930f-4f69-a798-ff9bfc84d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:50.889713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2b197-930f-4f69-a798-ff9bfc84d295 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:50.890346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2b197-930f-4f69-a798-ff9bfc84d295 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:50.890889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2b197-930f-4f69-a798-ff9bfc84d295 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:50.892018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2b197-930f-4f69-a798-ff9bfc84d295 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:50.892368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2b197-930f-4f69-a798-ff9bfc84d295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:50.893765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2b197-930f-4f69-a798-ff9bfc84d295 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:50.898846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 468/960] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:51.072915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a7c76-6e3d-4921-b8dc-1ebca5055179 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:51.073881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a7c76-6e3d-4921-b8dc-1ebca5055179 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:51.074367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a7c76-6e3d-4921-b8dc-1ebca5055179 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:51.076152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 493/961] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:51.081783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f20f0-5b3a-43d6-ae8a-9e2c5ed63354 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:51.082343 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f20f0-5b3a-43d6-ae8a-9e2c5ed63354 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:51.082821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f20f0-5b3a-43d6-ae8a-9e2c5ed63354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:51.102140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-447f20f0-5b3a-43d6-ae8a-9e2c5ed63354 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:51.103922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-447f20f0-5b3a-43d6-ae8a-9e2c5ed63354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:51.177096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 469/962] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:51.200254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a7c76-6e3d-4921-b8dc-1ebca5055179 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:51.257696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:51.311394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a7c76-6e3d-4921-b8dc-1ebca5055179 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:51.347779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e07833-3db1-41fc-bd67-31dc5ef17da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:51.348395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e07833-3db1-41fc-bd67-31dc5ef17da7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:51.348676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e07833-3db1-41fc-bd67-31dc5ef17da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:51.348946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e07833-3db1-41fc-bd67-31dc5ef17da7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:51.359703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e07833-3db1-41fc-bd67-31dc5ef17da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-08e07833-3db1-41fc-bd67-31dc5ef17da7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:24:51.365181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:51.366183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:51.367057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 494/963] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jul 28 06:24:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:51.372600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f20f0-5b3a-43d6-ae8a-9e2c5ed63354 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:51.481098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f20f0-5b3a-43d6-ae8a-9e2c5ed63354 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:51.495994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:51.517575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:51.518203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:51.518483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:51.518754 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:51.519466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:51.519767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:51.521122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:51.529801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 470/964] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jul 28 06:24:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:51.535877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e07833-3db1-41fc-bd67-31dc5ef17da7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:51.639294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e07833-3db1-41fc-bd67-31dc5ef17da7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:51.650524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:51.676202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:51.676852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:51.677130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:51.677425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:51.687889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:51.689578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:51.704890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:51.707378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:51.709541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:51.709541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:51.709541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:51.709541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:51.709541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 495/965] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:51.713318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:51.818710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91042307-71a9-4e60-8342-e536df93c1b1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:51.854113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:51.854734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:51.855014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:51.855288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:51.855972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:51.856273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:51.857650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:51.870207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 471/966] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:51.876231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:51.979338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f1c33-5ffb-4561-9292-c1216aab76d8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.017197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7c91a-9fa0-413c-a8f2-eaed47aa2d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.017867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7c91a-9fa0-413c-a8f2-eaed47aa2d66 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.018149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7c91a-9fa0-413c-a8f2-eaed47aa2d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.018421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7c91a-9fa0-413c-a8f2-eaed47aa2d66 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.026008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7c91a-9fa0-413c-a8f2-eaed47aa2d66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.027445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7c91a-9fa0-413c-a8f2-eaed47aa2d66 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-bab7c91a-9fa0-413c-a8f2-eaed47aa2d66 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:52.040861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:52.043096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:52.045394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 496/967] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:24:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:52.048744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.092362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:52.145056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34baf6-2d21-424a-9dcc-25f02353d28d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.145764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34baf6-2d21-424a-9dcc-25f02353d28d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.146555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34baf6-2d21-424a-9dcc-25f02353d28d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.148333 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 497/968] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:52.153625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e2be-9fcd-4257-8ba1-629e3345a4c1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.154155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e2be-9fcd-4257-8ba1-629e3345a4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.154645 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960e2be-9fcd-4257-8ba1-629e3345a4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.162308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89d448-6fb8-4d8a-9902-a00f0020ab38 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.174085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0960e2be-9fcd-4257-8ba1-629e3345a4c1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:52.175652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0960e2be-9fcd-4257-8ba1-629e3345a4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:52.197558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.197757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:52.198059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.198597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.198873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.199666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:52.199971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.201478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.224660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 472/969] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Jul 28 06:24:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:52.225739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.254958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 498/970] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:52.277455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960e2be-9fcd-4257-8ba1-629e3345a4c1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.321727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:52.327845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46141725-fd19-4604-bfa9-35a6ff82d749 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.365414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26581da0-51b2-47b9-96a0-38a1ccf61fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.366025 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26581da0-51b2-47b9-96a0-38a1ccf61fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.366304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26581da0-51b2-47b9-96a0-38a1ccf61fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.366576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26581da0-51b2-47b9-96a0-38a1ccf61fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.386097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960e2be-9fcd-4257-8ba1-629e3345a4c1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.424256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841e330-b5c2-4f87-912e-a59572d84a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.424902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e330-b5c2-4f87-912e-a59572d84a63 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.425182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e330-b5c2-4f87-912e-a59572d84a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.425490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e330-b5c2-4f87-912e-a59572d84a63 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.426120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841e330-b5c2-4f87-912e-a59572d84a63 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.427479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841e330-b5c2-4f87-912e-a59572d84a63 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.437311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26581da0-51b2-47b9-96a0-38a1ccf61fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.438978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26581da0-51b2-47b9-96a0-38a1ccf61fa9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4841e330-b5c2-4f87-912e-a59572d84a63 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:52.440553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:52.441607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:52.442481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:52.449952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 499/971] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:52.451439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb674f4-f333-4376-af51-65940d55dc3a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.451727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb674f4-f333-4376-af51-65940d55dc3a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.452002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb674f4-f333-4376-af51-65940d55dc3a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.457835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 500/972] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:52.458641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb674f4-f333-4376-af51-65940d55dc3a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.480357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:52.541652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:52.555847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 473/973] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:24:52 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:24:52.557153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b8b5c-aed3-4add-a1e7-fe4bfa1ce806 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.557471 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b8b5c-aed3-4add-a1e7-fe4bfa1ce806 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.557748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b8b5c-aed3-4add-a1e7-fe4bfa1ce806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.568717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb674f4-f333-4376-af51-65940d55dc3a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.577708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-467b8b5c-aed3-4add-a1e7-fe4bfa1ce806 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:52.579540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-467b8b5c-aed3-4add-a1e7-fe4bfa1ce806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:52.603991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.604619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.604955 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.605233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.605943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:52.606256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.607614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.612784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 501/974] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:52.613514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.655496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 474/975] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:52.656830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b8b5c-aed3-4add-a1e7-fe4bfa1ce806 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.712685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:52.736785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337b3eb-1293-45e5-ba60-4dd932ab71ed None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.771707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.772348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.772518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b8b5c-aed3-4add-a1e7-fe4bfa1ce806 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.772620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.772886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.784532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.786015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.796707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:24:52.801050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:52.802032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:52.802965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:24:52.802965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:24:52.802965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:24:52.802965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:52.802965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 502/976] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:52.802965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.814704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.815319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.815606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.815878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.816556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:52.816856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.818254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.828067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 475/977] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:52.828067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.936062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26488d-59f8-4316-8056-b2920b687b89 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.936268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79e46-408d-405d-9ccd-dc24140d087a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:52.968613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:52.974914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.975191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:52.975542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.975806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:52.975908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.976086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.976196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:52.976352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:52.976797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:52.977030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.977128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:52.978461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.978561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:52.981767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING keystone.server.flask.common [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 06:24:52.990101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 476/978] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:52.990966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:52.996652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.sql.core [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 06:24:52.996652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 06:24:52.996652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [parameters: {'user_id': '9cd200eaca96466982a1b26258ae6b40', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 06:24:52.996652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80277) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c2ba11fc-8206-476d-96d5-80d32bd504f9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result.read() Jul 28 06:24:53.005683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:24:53.006619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application yield resource Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application next(self.gen) Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise self.value Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application session.commit() Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 06:24:53.007623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.session.flush() Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application compat.raise_( Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 06:24:53.008520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 06:24:53.009449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application conn.query(q) Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result.read() Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 06:24:53.011367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [parameters: {'user_id': '9cd200eaca96466982a1b26258ae6b40', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:53.012784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 06:24:53.017969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 503/979] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Jul 28 06:24:52 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc93ea-7ccc-49bf-bcf6-b8b130a16dd6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc93ea-7ccc-49bf-bcf6-b8b130a16dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.018926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc93ea-7ccc-49bf-bcf6-b8b130a16dd6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.019522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 504/980] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:53.019522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc93ea-7ccc-49bf-bcf6-b8b130a16dd6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:53.049893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:53.102492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda3575-6a3b-4238-b75e-a9a450dcf226 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:53.115705 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc93ea-7ccc-49bf-bcf6-b8b130a16dd6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:53.137799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:53.138412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.138690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.138966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.139712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] RBAC: Adding query filter params (interface=admin, service_id=182615cceb7949e9a10867ac890f1e0e, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:53.140020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:53.142109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:53.146987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 477/981] 10.222.0.20 () {56 vars in 1249 bytes} [Thu Jul 28 06:24:52 2022] GET /identity/v3/endpoints?service_id=182615cceb7949e9a10867ac890f1e0e&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:53.147759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:53.152056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6510952-3b9b-4a9d-98d1-d1c026e8ac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:53.152705 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6510952-3b9b-4a9d-98d1-d1c026e8ac0f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.152992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6510952-3b9b-4a9d-98d1-d1c026e8ac0f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.153473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6510952-3b9b-4a9d-98d1-d1c026e8ac0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.161179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6510952-3b9b-4a9d-98d1-d1c026e8ac0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:53.162783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6510952-3b9b-4a9d-98d1-d1c026e8ac0f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e6510952-3b9b-4a9d-98d1-d1c026e8ac0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:24:53.175402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:53.177192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:53.179059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 505/982] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:53.180036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf627b-0650-40a2-9d0f-83084eab33bc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.180036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf627b-0650-40a2-9d0f-83084eab33bc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.180867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf627b-0650-40a2-9d0f-83084eab33bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.200001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2cdf627b-0650-40a2-9d0f-83084eab33bc None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:53.201529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2cdf627b-0650-40a2-9d0f-83084eab33bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:53.249751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:53.253783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1510ed-baf9-4df8-a869-e492ed6c4a9d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:53.266642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:53.268627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 506/983] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:53.269538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf627b-0650-40a2-9d0f-83084eab33bc None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:53.293456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:53.294101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.294380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.294649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.304942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:53.306362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:24:53.317868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:53.318864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 478/984] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:24:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:53.320388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:53.370695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf627b-0650-40a2-9d0f-83084eab33bc None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:53.404570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adde58-6e06-4493-ae43-99fc1e152c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:53.405185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adde58-6e06-4493-ae43-99fc1e152c63 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.405495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adde58-6e06-4493-ae43-99fc1e152c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.405779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adde58-6e06-4493-ae43-99fc1e152c63 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.406491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adde58-6e06-4493-ae43-99fc1e152c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:53.407870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adde58-6e06-4493-ae43-99fc1e152c63 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:53.414497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 507/985] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:24:53 2022] GET /identity/v3/services => generated 1497 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:24:53.415058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adde58-6e06-4493-ae43-99fc1e152c63 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:53.432929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bc1a0-02f7-4d21-94e1-93330fb0b92d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:53.477272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:53.484001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:53.485490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.485812 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.486056 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.486820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:53.487130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:53.489509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:53.505112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 479/986] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:53.506008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:53.542090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:53.546931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adde58-6e06-4493-ae43-99fc1e152c63 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:53.587338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810bf87-4d6e-4304-a013-2d9c7a221d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:53.587919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810bf87-4d6e-4304-a013-2d9c7a221d7f None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.588195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810bf87-4d6e-4304-a013-2d9c7a221d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.588466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810bf87-4d6e-4304-a013-2d9c7a221d7f None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.589161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810bf87-4d6e-4304-a013-2d9c7a221d7f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:53.590550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810bf87-4d6e-4304-a013-2d9c7a221d7f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:53.609144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 508/987] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:24:53 2022] POST /identity/v3/services => generated 242 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:53.610601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810bf87-4d6e-4304-a013-2d9c7a221d7f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:53.631929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8b61b-7126-48e7-85ae-c04812e22af1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:53.689968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365e409-8728-47a0-8079-428ec71a532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:53.690602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e409-8728-47a0-8079-428ec71a532c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.690878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e409-8728-47a0-8079-428ec71a532c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.691211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e409-8728-47a0-8079-428ec71a532c None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.692142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365e409-8728-47a0-8079-428ec71a532c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:53.692540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365e409-8728-47a0-8079-428ec71a532c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:53.694049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365e409-8728-47a0-8079-428ec71a532c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:53.736658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 480/988] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Jul 28 06:24:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:53.771345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810bf87-4d6e-4304-a013-2d9c7a221d7f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:53.783790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:53.816390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b565a-fc09-429c-b472-0bb041d94f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:53.817028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b565a-fc09-429c-b472-0bb041d94f1f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:53.817357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b565a-fc09-429c-b472-0bb041d94f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:53.817609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b565a-fc09-429c-b472-0bb041d94f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:53.818411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b565a-fc09-429c-b472-0bb041d94f1f None admin] RBAC: Adding query filter params (role.id=030852a56a8145e3b8b7c785b4ace178, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=528c54678648461397f490fbbb698601) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:53.818715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b565a-fc09-429c-b472-0bb041d94f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:53.820145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b565a-fc09-429c-b472-0bb041d94f1f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:53.827704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 509/989] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:24:53 2022] GET /identity/v3/role_assignments?user.id=528c54678648461397f490fbbb698601&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=030852a56a8145e3b8b7c785b4ace178 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:55.482302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230c8f-82d9-4d13-b96e-64ca8e54e7bd None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.482806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230c8f-82d9-4d13-b96e-64ca8e54e7bd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.482919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230c8f-82d9-4d13-b96e-64ca8e54e7bd None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.484374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 481/990] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:55.489648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880cc7-dfd1-456b-8229-a4dc6aef50d0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.490055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880cc7-dfd1-456b-8229-a4dc6aef50d0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.490222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880cc7-dfd1-456b-8229-a4dc6aef50d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.508741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6d880cc7-dfd1-456b-8229-a4dc6aef50d0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:55.509968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6d880cc7-dfd1-456b-8229-a4dc6aef50d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:55.581121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 510/991] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:55.604263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0230c8f-82d9-4d13-b96e-64ca8e54e7bd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:55.690987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30be15-55ce-4f21-ad08-108757b0f1ce None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.691280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30be15-55ce-4f21-ad08-108757b0f1ce None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.691556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30be15-55ce-4f21-ad08-108757b0f1ce None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.692481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:55.692919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 511/992] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:55.698602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90799f92-7cce-4383-9db0-720d170891a5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.698893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90799f92-7cce-4383-9db0-720d170891a5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.699168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90799f92-7cce-4383-9db0-720d170891a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.712726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0230c8f-82d9-4d13-b96e-64ca8e54e7bd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:55.719636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-90799f92-7cce-4383-9db0-720d170891a5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:55.720847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-90799f92-7cce-4383-9db0-720d170891a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:55.755254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:55.755900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.756184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.756460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.757087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:55.758604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:55.768039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:55.769079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:55.770409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 482/993] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Jul 28 06:24:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:24:55.775209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:55.798159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 512/994] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:55.821026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90799f92-7cce-4383-9db0-720d170891a5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:55.854710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:55.881544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cd66f-4712-4ea6-88c0-e536cfbc1a68 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:55.915115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9faea4-b695-4208-8225-2a3340deac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:55.915735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9faea4-b695-4208-8225-2a3340deac10 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.916020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9faea4-b695-4208-8225-2a3340deac10 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.916299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9faea4-b695-4208-8225-2a3340deac10 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.916995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9faea4-b695-4208-8225-2a3340deac10 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:55.917319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9faea4-b695-4208-8225-2a3340deac10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:55.918696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9faea4-b695-4208-8225-2a3340deac10 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:55.923617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 483/995] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:55.924759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932203c4-461d-4dde-8624-871fd0823f45 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.924977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90799f92-7cce-4383-9db0-720d170891a5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:55.925083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932203c4-461d-4dde-8624-871fd0823f45 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.925332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932203c4-461d-4dde-8624-871fd0823f45 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.926791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 484/996] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:55.929876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932203c4-461d-4dde-8624-871fd0823f45 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:55.945374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:55.961319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fb219-226a-4ac6-b4a5-19e935e1dd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:55.961997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fb219-226a-4ac6-b4a5-19e935e1dd9e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.962281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fb219-226a-4ac6-b4a5-19e935e1dd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.962580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fb219-226a-4ac6-b4a5-19e935e1dd9e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.963262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fb219-226a-4ac6-b4a5-19e935e1dd9e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:55.964565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fb219-226a-4ac6-b4a5-19e935e1dd9e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d14fb219-226a-4ac6-b4a5-19e935e1dd9e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:55.973903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:55.974989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:55.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 513/997] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:55.977088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ef84-73e8-4bf2-9b78-56f821124a28 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:55.977088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ef84-73e8-4bf2-9b78-56f821124a28 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:55.977088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ef84-73e8-4bf2-9b78-56f821124a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:55.994279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-49f3ef84-73e8-4bf2-9b78-56f821124a28 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:55.995488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-49f3ef84-73e8-4bf2-9b78-56f821124a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:56.035536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932203c4-461d-4dde-8624-871fd0823f45 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.062289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 514/998] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:56.063706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3ef84-73e8-4bf2-9b78-56f821124a28 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:56.070719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.071743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.072037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.072305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.072984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:56.073916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.075802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.087962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 485/999] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Jul 28 06:24:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:56.088914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:56.091289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:56.166911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3ef84-73e8-4bf2-9b78-56f821124a28 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.171675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:56.194660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e980a-edc1-4262-aa8c-2593fe0b643f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.201885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.202501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.202783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.203776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.204875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:56.205181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.206582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.211277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 515/1000] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:56.212029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:56.228434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.229073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.229385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.229655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] PATH_INFO: `/v3/services/image` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.230276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.231659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:56.240688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:56.241891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:24:56.243280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 486/1001] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Jul 28 06:24:56 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:56.244110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:56.313744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:56.324701 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2341781-9950-458f-acbb-845a0d922108 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.342068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb71762-4c37-416b-9182-ab2131103e9d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.360235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d76e0-dbe0-4196-baa7-179842ab28bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.360845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d76e0-dbe0-4196-baa7-179842ab28bc None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.361124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d76e0-dbe0-4196-baa7-179842ab28bc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.361429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d76e0-dbe0-4196-baa7-179842ab28bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.362129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d76e0-dbe0-4196-baa7-179842ab28bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.363479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d76e0-dbe0-4196-baa7-179842ab28bc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.376225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.376841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.377119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.377421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.378094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:56.378399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.379742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.380253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 516/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:24:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:56.381208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d76e0-dbe0-4196-baa7-179842ab28bc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:56.384818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 487/1003] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:56.393127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:56.403227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:56.518572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9b0f-1063-49bc-8dd1-6d484c292333 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.532281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d76e0-dbe0-4196-baa7-179842ab28bc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.561522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.563136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.563136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.563492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.564481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] RBAC: Adding query filter params (interface=public, service_id=b496242c4b5d4ce9b8f49ece1c1da6d5, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:56.564481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.565848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.571822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.572446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.573569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.573569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.573965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] RBAC: Adding query filter params (name=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:56.574270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.575620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.577384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 488/1004] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jul 28 06:24:56 2022] GET /identity/v3/endpoints?service_id=b496242c4b5d4ce9b8f49ece1c1da6d5&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:56.579373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:56.581458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 517/1005] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:56.587191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:56.601469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:56.652993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:56.688245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f38c6-30a1-4c34-9f69-386fd49e6776 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.700234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6af94-167c-409e-9d12-7c20bf1cf45f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:56.723705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2676d4d-6123-4ad9-9a61-3f566c509dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.724333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676d4d-6123-4ad9-9a61-3f566c509dbd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.724616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676d4d-6123-4ad9-9a61-3f566c509dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.724894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676d4d-6123-4ad9-9a61-3f566c509dbd None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.725577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2676d4d-6123-4ad9-9a61-3f566c509dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.726949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2676d4d-6123-4ad9-9a61-3f566c509dbd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.733142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 489/1006] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:24:56 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:24:56.736384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13985e-c015-4fdf-b88f-f29e8b0d2c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:56.737015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13985e-c015-4fdf-b88f-f29e8b0d2c9e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:56.737307 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13985e-c015-4fdf-b88f-f29e8b0d2c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:56.737594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13985e-c015-4fdf-b88f-f29e8b0d2c9e None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:56.738279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13985e-c015-4fdf-b88f-f29e8b0d2c9e None admin] RBAC: Adding query filter params (type=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:56.738607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13985e-c015-4fdf-b88f-f29e8b0d2c9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:56.739995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13985e-c015-4fdf-b88f-f29e8b0d2c9e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:56.745747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 518/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:58.410825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc98b-9fbc-4169-876e-364054af7f76 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:58.411381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc98b-9fbc-4169-876e-364054af7f76 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:58.411381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc98b-9fbc-4169-876e-364054af7f76 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:58.412867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 490/1008] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:58.418057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda09d-49cd-40ba-bf4a-c853c44b827a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:58.418437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda09d-49cd-40ba-bf4a-c853c44b827a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:58.418614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda09d-49cd-40ba-bf4a-c853c44b827a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:58.439582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1ceda09d-49cd-40ba-bf4a-c853c44b827a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:58.440772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1ceda09d-49cd-40ba-bf4a-c853c44b827a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:58.517492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 519/1009] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:58.536757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bc98b-9fbc-4169-876e-364054af7f76 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:58.643898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bc98b-9fbc-4169-876e-364054af7f76 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:58.678577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15594d91-9975-41f1-9276-81fa1e02d0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:58.679135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594d91-9975-41f1-9276-81fa1e02d0ef None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:58.679471 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594d91-9975-41f1-9276-81fa1e02d0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:58.679747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594d91-9975-41f1-9276-81fa1e02d0ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:58.680939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15594d91-9975-41f1-9276-81fa1e02d0ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:58.682402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15594d91-9975-41f1-9276-81fa1e02d0ef None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:58.684204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 491/1010] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:24:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:58.690774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda09d-49cd-40ba-bf4a-c853c44b827a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:58.693950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:58.792741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda09d-49cd-40ba-bf4a-c853c44b827a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:58.807754 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b7db-747f-4542-b1c7-feb0c0e9c686 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:58.808039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b7db-747f-4542-b1c7-feb0c0e9c686 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:58.808313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b7db-747f-4542-b1c7-feb0c0e9c686 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:58.809693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 492/1011] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:24:58.810692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e3e3-c33b-4e60-bbef-004496dd4b14 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:58.810972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e3e3-c33b-4e60-bbef-004496dd4b14 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:58.811245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e3e3-c33b-4e60-bbef-004496dd4b14 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:58.813018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 493/1012] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:24:58.817893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1081608-b299-4f41-98ff-8944b111a0e3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:58.818178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1081608-b299-4f41-98ff-8944b111a0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:58.818455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1081608-b299-4f41-98ff-8944b111a0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:58.824926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:58.831254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f148c73-28a8-4fcb-bc1c-4649d64c9358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:58.831875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148c73-28a8-4fcb-bc1c-4649d64c9358 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:58.832156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148c73-28a8-4fcb-bc1c-4649d64c9358 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:58.832435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148c73-28a8-4fcb-bc1c-4649d64c9358 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:58.833129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f148c73-28a8-4fcb-bc1c-4649d64c9358 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:58.834645 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f148c73-28a8-4fcb-bc1c-4649d64c9358 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:58.837872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d1081608-b299-4f41-98ff-8944b111a0e3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:58.839037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d1081608-b299-4f41-98ff-8944b111a0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:58.863251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 520/1013] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:24:58 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:24:58.864704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b672ae-b45a-43a1-a032-eccc09a700ab None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:58.865001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b672ae-b45a-43a1-a032-eccc09a700ab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:58.865284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b672ae-b45a-43a1-a032-eccc09a700ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:58.886817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-59b672ae-b45a-43a1-a032-eccc09a700ab None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:24:58.888627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-59b672ae-b45a-43a1-a032-eccc09a700ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:24:58.923005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 494/1014] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:24:58.946590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1081608-b299-4f41-98ff-8944b111a0e3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:58.961174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:58.982417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 521/1015] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:24:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:24:59.006767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b672ae-b45a-43a1-a032-eccc09a700ab None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:59.064264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1081608-b299-4f41-98ff-8944b111a0e3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:59.097002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:59.103824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edf468-7423-4c85-9e3e-b0a70ef4b2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:59.104450 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edf468-7423-4c85-9e3e-b0a70ef4b2f8 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:59.104733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edf468-7423-4c85-9e3e-b0a70ef4b2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:59.105006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edf468-7423-4c85-9e3e-b0a70ef4b2f8 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:59.105736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edf468-7423-4c85-9e3e-b0a70ef4b2f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:59.107118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edf468-7423-4c85-9e3e-b0a70ef4b2f8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:59.123026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b672ae-b45a-43a1-a032-eccc09a700ab None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:59.125260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 495/1016] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:24:58 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:24:59.171144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:59.172412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:59.172857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:59.173193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] PATH_INFO: `/v3/services/compute` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:59.173911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:59.175317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:24:59.185178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:24:59.186457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:24:59.187498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 522/1017] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:24:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:24:59.193438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edf468-7423-4c85-9e3e-b0a70ef4b2f8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:59.303832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edf468-7423-4c85-9e3e-b0a70ef4b2f8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:59.339263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:59.339884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:59.340163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:59.340436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:59.341112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:59.341495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:59.342855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:59.353484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 496/1018] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:24:59.355407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:59.405627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:59.471325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2739f36-c7cd-44ae-936a-57fa70c87cab None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:59.506879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89331a15-c519-4e4f-9ba1-d1bdfa43ee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:59.507444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89331a15-c519-4e4f-9ba1-d1bdfa43ee02 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:59.507721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89331a15-c519-4e4f-9ba1-d1bdfa43ee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:59.507988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89331a15-c519-4e4f-9ba1-d1bdfa43ee02 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:59.508654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89331a15-c519-4e4f-9ba1-d1bdfa43ee02 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:24:59.508953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89331a15-c519-4e4f-9ba1-d1bdfa43ee02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:59.510327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89331a15-c519-4e4f-9ba1-d1bdfa43ee02 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:59.515477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 523/1019] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Jul 28 06:24:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:24:59.521189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:24:59.581311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:24:59.628751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31d0b8-5951-4231-9ac2-6fefc5f6d0b8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:24:59.662427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0ad5f-5889-433f-9d51-458283d9e78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:24:59.663048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0ad5f-5889-433f-9d51-458283d9e78d None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:24:59.663329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0ad5f-5889-433f-9d51-458283d9e78d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:24:59.663603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0ad5f-5889-433f-9d51-458283d9e78d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:24:59.664319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0ad5f-5889-433f-9d51-458283d9e78d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:24:59.665782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0ad5f-5889-433f-9d51-458283d9e78d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:24:59.693370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 497/1020] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:24:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:25:00.934115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cafd7b-2a8b-4454-aa69-694910c00ea2 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:00.934636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cafd7b-2a8b-4454-aa69-694910c00ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:00.934742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cafd7b-2a8b-4454-aa69-694910c00ea2 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:00.936101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 524/1021] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:00.941807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144de5e-b5bd-4fa4-a226-46c0c25bf2dd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:00.942241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144de5e-b5bd-4fa4-a226-46c0c25bf2dd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:00.942453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144de5e-b5bd-4fa4-a226-46c0c25bf2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:00.961157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8144de5e-b5bd-4fa4-a226-46c0c25bf2dd None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:00.962386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8144de5e-b5bd-4fa4-a226-46c0c25bf2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:01.037589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 498/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:01.062022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cafd7b-2a8b-4454-aa69-694910c00ea2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:01.165574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cafd7b-2a8b-4454-aa69-694910c00ea2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:01.173620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c6668-a914-4a01-abb6-4c9d1f911b50 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.173914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c6668-a914-4a01-abb6-4c9d1f911b50 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.174199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c6668-a914-4a01-abb6-4c9d1f911b50 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.175796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 499/1023] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:01.177650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:01.180679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78138-a95e-4291-abce-e8548f2877ba None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.181002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78138-a95e-4291-abce-e8548f2877ba None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.181314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78138-a95e-4291-abce-e8548f2877ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.201159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-55b78138-a95e-4291-abce-e8548f2877ba None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:01.202292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-55b78138-a95e-4291-abce-e8548f2877ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:01.205116 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea145d-cf28-4606-8414-d5d20c703f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:01.205803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea145d-cf28-4606-8414-d5d20c703f2c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.206089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea145d-cf28-4606-8414-d5d20c703f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.206369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea145d-cf28-4606-8414-d5d20c703f2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.207528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea145d-cf28-4606-8414-d5d20c703f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:01.208965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea145d-cf28-4606-8414-d5d20c703f2c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:01.210605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 525/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:01.216324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea145d-cf28-4606-8414-d5d20c703f2c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:01.240481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:01.281682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 500/1025] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:25:01.304659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78138-a95e-4291-abce-e8548f2877ba None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:01.325938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea145d-cf28-4606-8414-d5d20c703f2c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:01.363208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb843c-d728-4e35-b8c6-750571e34b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:01.363823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb843c-d728-4e35-b8c6-750571e34b67 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.364095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb843c-d728-4e35-b8c6-750571e34b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.364388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb843c-d728-4e35-b8c6-750571e34b67 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.365246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb843c-d728-4e35-b8c6-750571e34b67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:01.366486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb843c-d728-4e35-b8c6-750571e34b67 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:01.390712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 526/1026] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:25:01 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:25:01.425249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78138-a95e-4291-abce-e8548f2877ba None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:01.475016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:01.475890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.476270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.476646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] PATH_INFO: `/v3/services/image` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.477422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:01.478919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:01.491085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:01.492264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:01.493167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 501/1027] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Jul 28 06:25:01 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:25:01.498808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb843c-d728-4e35-b8c6-750571e34b67 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:01.542835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:01.620833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb843c-d728-4e35-b8c6-750571e34b67 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:01.667282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e3465-c35b-42ad-84b6-5f04548c5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:01.667846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e3465-c35b-42ad-84b6-5f04548c5df9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.668134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e3465-c35b-42ad-84b6-5f04548c5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.668412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e3465-c35b-42ad-84b6-5f04548c5df9 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.669100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e3465-c35b-42ad-84b6-5f04548c5df9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:01.670221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e3465-c35b-42ad-84b6-5f04548c5df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:01.672131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e3465-c35b-42ad-84b6-5f04548c5df9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:01.677944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 527/1028] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:01.690634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:01.724756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:01.798272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee20e4-5c55-4cf3-b765-60c97dad0c90 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.798605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee20e4-5c55-4cf3-b765-60c97dad0c90 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.798824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee20e4-5c55-4cf3-b765-60c97dad0c90 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.798927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d463317-a509-427b-badc-cff8be40b40a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:01.800299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 528/1029] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:25:01.805083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a24f6-1470-4d5f-aae2-89c83d0cd506 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.805396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a24f6-1470-4d5f-aae2-89c83d0cd506 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.805671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a24f6-1470-4d5f-aae2-89c83d0cd506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.824058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d79a24f6-1470-4d5f-aae2-89c83d0cd506 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:01.825244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d79a24f6-1470-4d5f-aae2-89c83d0cd506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:01.838720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:01.839344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.839622 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.840001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:01.840788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] RBAC: Adding query filter params (type=image) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:01.841094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:01.842511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:01.853729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 502/1030] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:01.854729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:01.896657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 529/1031] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:25:01.920716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a24f6-1470-4d5f-aae2-89c83d0cd506 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:01.950366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:01.954929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:01.961745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc1c9-28bb-48f3-862c-db9fb638943e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:01.997571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:01.998554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:01.999054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:01.999544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:02.001199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] RBAC: Adding query filter params (interface=public, service_id=007920a8976a43f4baf7c24d05bc8d2c, region_id=RegionOne) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:02.001690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:02.003345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:02.013348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 503/1032] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jul 28 06:25:01 2022] GET /identity/v3/endpoints?service_id=007920a8976a43f4baf7c24d05bc8d2c&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:02.014385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:02.043409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a24f6-1470-4d5f-aae2-89c83d0cd506 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:02.084238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3c6cc-2ac9-4146-8e2f-aeb6b8ef7181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:02.084881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3c6cc-2ac9-4146-8e2f-aeb6b8ef7181 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:02.085164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3c6cc-2ac9-4146-8e2f-aeb6b8ef7181 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:02.085474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3c6cc-2ac9-4146-8e2f-aeb6b8ef7181 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:02.096172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3c6cc-2ac9-4146-8e2f-aeb6b8ef7181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7ae3c6cc-2ac9-4146-8e2f-aeb6b8ef7181 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:02.101231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:02.102437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:02.106529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 530/1033] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jul 28 06:25:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:25:02.108326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3c6cc-2ac9-4146-8e2f-aeb6b8ef7181 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:02.129615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f432e-e55b-4e10-b451-a2d81d4330c4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:02.169135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb30a15-5eca-4e03-8ac6-8fad193c7939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:02.170066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb30a15-5eca-4e03-8ac6-8fad193c7939 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:02.170506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb30a15-5eca-4e03-8ac6-8fad193c7939 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:02.170941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb30a15-5eca-4e03-8ac6-8fad193c7939 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:02.172156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb30a15-5eca-4e03-8ac6-8fad193c7939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:02.173555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb30a15-5eca-4e03-8ac6-8fad193c7939 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:02.180345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 504/1034] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:25:02 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:25:02.208421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:02.224649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3c6cc-2ac9-4146-8e2f-aeb6b8ef7181 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:02.264610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:02.265158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:02.266016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:02.266695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:02.267043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:02.267349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:02.268740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:02.283388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 531/1035] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jul 28 06:25:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:02.285060 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb30a15-5eca-4e03-8ac6-8fad193c7939 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:02.315462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:02.418198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb30a15-5eca-4e03-8ac6-8fad193c7939 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:02.461717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:02.462370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:02.462656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:02.462934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:02.474185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:02.475641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:25:02.490864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:02.491811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:02.497564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:25:02.497564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:25:02.497564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:02.497564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:02.497564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 505/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:02.499342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:02.605379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126fcba-fc0c-409c-8ec1-6aa816251360 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:02.642856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:02.643701 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:02.644101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:02.644478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:02.645279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:02.645710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:02.647169 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:02.665025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 532/1037] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:02.666303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:02.748538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:02.773618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a08e9-575a-4de4-8194-98191b388419 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:02.806245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:02.806951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:02.807301 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:02.807642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:02.815458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:02.816964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:25:02.828961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:02.830176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:25:02.831377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:02.833641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 506/1038] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:25:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:02.837267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:02.850648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:02.943321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb61656-dce1-4836-b208-f971e2fba01a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:02.979315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:02.979947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:02.980206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:02.980480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:02.981270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:02.981604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:02.983019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:03.005522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 533/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:25:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:03.006713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:03.113104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324b07-ad71-4a6a-b82f-82680d609cc4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:03.149378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614013c-98fd-401b-bd4d-5e01fc4108b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:03.150003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614013c-98fd-401b-bd4d-5e01fc4108b3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:03.150286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614013c-98fd-401b-bd4d-5e01fc4108b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:03.150560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614013c-98fd-401b-bd4d-5e01fc4108b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:03.154066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:03.222693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614013c-98fd-401b-bd4d-5e01fc4108b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:03.224158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614013c-98fd-401b-bd4d-5e01fc4108b3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:03.329921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 507/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:03.331734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:03.407432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:03.446161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6cad9-131e-4b95-98d1-7d7bbe4b2b40 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:03.486308 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:03.486866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:03.487149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:03.487421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:03.497225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565539e-4e6c-4743-9aec-b062a3c5d03c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:03.497548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565539e-4e6c-4743-9aec-b062a3c5d03c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:03.497829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565539e-4e6c-4743-9aec-b062a3c5d03c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:03.498479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:03.499373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 508/1041] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:03.500443 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:03.504764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344fb8d-4d06-4749-b2de-71f124745892 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:03.505054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344fb8d-4d06-4749-b2de-71f124745892 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:03.505445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344fb8d-4d06-4749-b2de-71f124745892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:25:03.516205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:03.517345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:03.523178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:25:03.523178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:25:03.523178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:03.523178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:03.523178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 534/1042] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:03.525065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:03.529867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e344fb8d-4d06-4749-b2de-71f124745892 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:03.531691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e344fb8d-4d06-4749-b2de-71f124745892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:03.561240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:03.600243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 509/1043] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:25:03.623648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344fb8d-4d06-4749-b2de-71f124745892 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:03.634283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c5a6a-75f6-497d-8966-975092ec2ebe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:03.654711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:03.673161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:03.673802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:03.674081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:03.674349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:03.675121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:03.675327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:03.676678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:03.689078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 535/1044] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:03.694930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:03.739092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344fb8d-4d06-4749-b2de-71f124745892 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:03.776868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f5c27-8431-43f1-8310-b87cf2c41350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:03.777554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5c27-8431-43f1-8310-b87cf2c41350 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:03.777839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5c27-8431-43f1-8310-b87cf2c41350 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:03.778110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5c27-8431-43f1-8310-b87cf2c41350 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:03.789073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f5c27-8431-43f1-8310-b87cf2c41350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-632f5c27-8431-43f1-8310-b87cf2c41350 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:03.793998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:03.795041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:03.796046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 510/1045] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:03.801449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f5c27-8431-43f1-8310-b87cf2c41350 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:03.810149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270580cd-9c29-4533-8cb2-b742cbd3c708 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:03.839371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:03.847851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:03.848509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:03.848792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:03.849073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:03.856142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:03.857618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:25:03.869456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:03.870393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:03.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 536/1046] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:25:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:25:03.876936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:03.922434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f5c27-8431-43f1-8310-b87cf2c41350 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:03.924311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:03.960003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:03.960623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:03.960904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:03.961180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:03.961921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:03.962222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:03.963592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:03.972641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 511/1047] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:03.977974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:03.989539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a4608-1545-48ca-820f-75381ff106cf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.024377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.024997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.025281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.025581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.026327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:04.026636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.028052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.040192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 537/1048] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:25:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:04.047350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:04.075797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:04.090888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ffc91-652a-427a-ab0c-2fe34ba50563 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.133102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a7799-7c83-475a-8a03-4395bbe31ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.134292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a7799-7c83-475a-8a03-4395bbe31ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.134577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a7799-7c83-475a-8a03-4395bbe31ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.135108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a7799-7c83-475a-8a03-4395bbe31ba0 None admin] PATH_INFO: `/v3/users/dd06efffc1fb499f84307f61c7ced0ba` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.149655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a7799-7c83-475a-8a03-4395bbe31ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=dd06efffc1fb499f84307f61c7ced0ba)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.151110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a7799-7c83-475a-8a03-4395bbe31ba0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.152502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:04.152899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 512/1049] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:03 2022] GET /identity/v3/users/dd06efffc1fb499f84307f61c7ced0ba => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:04.159088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a7799-7c83-475a-8a03-4395bbe31ba0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:04.160131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd0f83-6fb8-4fa6-aa87-ee248e28c089 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.199178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df5477-57c7-4d65-a053-0cf792b04c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.199793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5477-57c7-4d65-a053-0cf792b04c9c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.200074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5477-57c7-4d65-a053-0cf792b04c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.200348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5477-57c7-4d65-a053-0cf792b04c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.201129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df5477-57c7-4d65-a053-0cf792b04c9c None admin] RBAC: Adding query filter params (role.id=030852a56a8145e3b8b7c785b4ace178, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=9f558687eefe4a26beb7d69151221660) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:04.201476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df5477-57c7-4d65-a053-0cf792b04c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.203073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df5477-57c7-4d65-a053-0cf792b04c9c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.208772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 538/1050] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:04 2022] GET /identity/v3/role_assignments?user.id=9f558687eefe4a26beb7d69151221660&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=030852a56a8145e3b8b7c785b4ace178 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:04.272444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a7799-7c83-475a-8a03-4395bbe31ba0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.300487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b57b21-5743-4faf-b85f-b851d50855c6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.300823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b57b21-5743-4faf-b85f-b851d50855c6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.301045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b57b21-5743-4faf-b85f-b851d50855c6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.302456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 539/1051] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:25:04.307938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1e977-0622-4b6c-8816-f48b05118592 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.308244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1e977-0622-4b6c-8816-f48b05118592 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.308500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1e977-0622-4b6c-8816-f48b05118592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.310752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943816ba-d3df-41b6-9458-9659e555288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.311384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943816ba-d3df-41b6-9458-9659e555288a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.311670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943816ba-d3df-41b6-9458-9659e555288a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.311943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943816ba-d3df-41b6-9458-9659e555288a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.330646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:04.332234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e7e1e977-0622-4b6c-8816-f48b05118592 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:04.333501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e7e1e977-0622-4b6c-8816-f48b05118592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:04.394080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943816ba-d3df-41b6-9458-9659e555288a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.395528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943816ba-d3df-41b6-9458-9659e555288a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.407958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 540/1052] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:25:04.435371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1e977-0622-4b6c-8816-f48b05118592 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:04.512651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 513/1053] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:04 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 06:25:04.519753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943816ba-d3df-41b6-9458-9659e555288a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:04.538843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:04.548743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1e977-0622-4b6c-8816-f48b05118592 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.568379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:04.594042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.594694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.594980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.595255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] PATH_INFO: `/v3/services/image` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.595888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.597259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:04.607879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:04.608917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 06:25:04.610201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:04.611481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 541/1054] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Jul 28 06:25:04 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:25:04.615675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:04.644338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943816ba-d3df-41b6-9458-9659e555288a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.689705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219b1d2-9c17-4a2e-9b21-8770feaae054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.690709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219b1d2-9c17-4a2e-9b21-8770feaae054 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.691113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219b1d2-9c17-4a2e-9b21-8770feaae054 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.691523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219b1d2-9c17-4a2e-9b21-8770feaae054 None admin] PATH_INFO: `/v3/projects/af99cb67bf83485794341607776e18c3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.693005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219b1d2-9c17-4a2e-9b21-8770feaae054 None admin] RBAC: Authorizing `identity:get_project(project_id=af99cb67bf83485794341607776e18c3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.694616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219b1d2-9c17-4a2e-9b21-8770feaae054 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.696624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 514/1055] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:04 2022] GET /identity/v3/projects/af99cb67bf83485794341607776e18c3 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:04.702491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219b1d2-9c17-4a2e-9b21-8770feaae054 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:04.728961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb2c5e-88ee-4fac-b389-97c13527175b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.767181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.767827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.768087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.768369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.769053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] RBAC: Adding query filter params (name=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:04.769402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.770815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.776034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 542/1056] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:04.782032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:04.807134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:04.819434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219b1d2-9c17-4a2e-9b21-8770feaae054 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.835280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:04.858537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308252aa-514f-4f54-9058-67638e252419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.859678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308252aa-514f-4f54-9058-67638e252419 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.860023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308252aa-514f-4f54-9058-67638e252419 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.860290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308252aa-514f-4f54-9058-67638e252419 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.861041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308252aa-514f-4f54-9058-67638e252419 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=af99cb67bf83485794341607776e18c3, user.id=dd06efffc1fb499f84307f61c7ced0ba) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:04.861379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308252aa-514f-4f54-9058-67638e252419 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.863478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308252aa-514f-4f54-9058-67638e252419 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.868560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 515/1057] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:04 2022] GET /identity/v3/role_assignments?user.id=dd06efffc1fb499f84307f61c7ced0ba&scope.project.id=af99cb67bf83485794341607776e18c3&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:04.897794 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8e363-265c-4446-a068-01f9362fb728 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:04.937091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b665853-5427-4de2-bd5b-ed8098364f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:04.938278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665853-5427-4de2-bd5b-ed8098364f48 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:04.938564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665853-5427-4de2-bd5b-ed8098364f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:04.938833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665853-5427-4de2-bd5b-ed8098364f48 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:04.939520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b665853-5427-4de2-bd5b-ed8098364f48 None admin] RBAC: Adding query filter params (type=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:04.939824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b665853-5427-4de2-bd5b-ed8098364f48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:04.941220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b665853-5427-4de2-bd5b-ed8098364f48 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:04.952153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 543/1058] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:04.953249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308252aa-514f-4f54-9058-67638e252419 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:05.063438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308252aa-514f-4f54-9058-67638e252419 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:05.098991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3bb2b-fa2a-411a-8a95-f4f5640aeaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:05.100362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3bb2b-fa2a-411a-8a95-f4f5640aeaed None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:05.100849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3bb2b-fa2a-411a-8a95-f4f5640aeaed None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:05.101366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3bb2b-fa2a-411a-8a95-f4f5640aeaed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:05.102293 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3bb2b-fa2a-411a-8a95-f4f5640aeaed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:05.103860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3bb2b-fa2a-411a-8a95-f4f5640aeaed None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:05.131498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 516/1059] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:25:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:25:06.219759 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6162-dbdb-4530-afe2-483139d6ef36 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:06.220291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6162-dbdb-4530-afe2-483139d6ef36 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:06.220291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6162-dbdb-4530-afe2-483139d6ef36 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:06.221709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 544/1060] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:06.226751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d0aa-3c83-41ce-9864-c8d1cee2aec7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:06.227155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d0aa-3c83-41ce-9864-c8d1cee2aec7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:06.227328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d0aa-3c83-41ce-9864-c8d1cee2aec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:06.245371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-22e5d0aa-3c83-41ce-9864-c8d1cee2aec7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:06.246532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-22e5d0aa-3c83-41ce-9864-c8d1cee2aec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:06.275122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:06.325156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 517/1061] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:06.348574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6162-dbdb-4530-afe2-483139d6ef36 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:06.442098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:06.463779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6162-dbdb-4530-afe2-483139d6ef36 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:06.500767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea8a01-175d-4671-b5b8-72773857bcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:06.501440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea8a01-175d-4671-b5b8-72773857bcbb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:06.501726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea8a01-175d-4671-b5b8-72773857bcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:06.502001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea8a01-175d-4671-b5b8-72773857bcbb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:06.512128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea8a01-175d-4671-b5b8-72773857bcbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-02ea8a01-175d-4671-b5b8-72773857bcbb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:06.518246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:06.519197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:06.520072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 545/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jul 28 06:25:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:06.527269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5d0aa-3c83-41ce-9864-c8d1cee2aec7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:06.630159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5d0aa-3c83-41ce-9864-c8d1cee2aec7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:06.666572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:06.667208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:06.667490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:06.667764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:06.668487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:06.668796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:06.670189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:06.680826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 518/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jul 28 06:25:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:06.685485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea8a01-175d-4671-b5b8-72773857bcbb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:06.788767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea8a01-175d-4671-b5b8-72773857bcbb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:06.823027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c908ec-2986-45ca-95b2-0ea9e955e2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:06.823920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c908ec-2986-45ca-95b2-0ea9e955e2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:06.824196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c908ec-2986-45ca-95b2-0ea9e955e2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:06.824463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c908ec-2986-45ca-95b2-0ea9e955e2e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:06.834773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c908ec-2986-45ca-95b2-0ea9e955e2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:06.836203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c908ec-2986-45ca-95b2-0ea9e955e2e1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:06.842995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-25c908ec-2986-45ca-95b2-0ea9e955e2e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:25:06.853082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:06.854073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:06.854998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:25:06.854998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:25:06.854998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:06.854998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:06.854998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 546/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:06.861757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:06.887823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:06.888249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e934c9-ac32-4fd0-8810-7435a3b9d8ab None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:06.888537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e934c9-ac32-4fd0-8810-7435a3b9d8ab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:06.888812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e934c9-ac32-4fd0-8810-7435a3b9d8ab None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:06.890911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 547/1065] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:06.896172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6641be-54c2-4fa0-83e5-1999618470c9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:06.896455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6641be-54c2-4fa0-83e5-1999618470c9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:06.896726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6641be-54c2-4fa0-83e5-1999618470c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:06.916162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9a6641be-54c2-4fa0-83e5-1999618470c9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:06.917465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9a6641be-54c2-4fa0-83e5-1999618470c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:06.968391 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1753f-ca4c-4427-87d7-20fb5759cf33 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:06.989330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 548/1066] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:25:07.008199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.008894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.009241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.009622 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.010472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:07.010776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.011569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6641be-54c2-4fa0-83e5-1999618470c9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.012134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.024534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 519/1067] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:07.029948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.118566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:07.125588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6641be-54c2-4fa0-83e5-1999618470c9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.143943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd623e-c7de-44ee-a755-cecf601eca18 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.158002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:07.166721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b86a-f36d-46f0-aa98-8093840c06d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.167407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b86a-f36d-46f0-aa98-8093840c06d9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.167683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b86a-f36d-46f0-aa98-8093840c06d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.167971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b86a-f36d-46f0-aa98-8093840c06d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.178917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b86a-f36d-46f0-aa98-8093840c06d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:07.182297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363eb56-0142-4b23-b32f-9ff58261dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.183153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363eb56-0142-4b23-b32f-9ff58261dddb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.183616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363eb56-0142-4b23-b32f-9ff58261dddb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4fa4b86a-f36d-46f0-aa98-8093840c06d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:07.183795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:07.184710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:07.185680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 549/1068] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:07.186424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363eb56-0142-4b23-b32f-9ff58261dddb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.191851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b86a-f36d-46f0-aa98-8093840c06d9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.194001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363eb56-0142-4b23-b32f-9ff58261dddb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.195647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363eb56-0142-4b23-b32f-9ff58261dddb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-9363eb56-0142-4b23-b32f-9ff58261dddb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:25:07.210097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:07.211134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:07.212120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 520/1069] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:25:07.213392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7537e71-ed28-48bd-a390-66097a8ff3db None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.213677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7537e71-ed28-48bd-a390-66097a8ff3db None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.213951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7537e71-ed28-48bd-a390-66097a8ff3db None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.215333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 521/1070] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:25:07.216732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7537e71-ed28-48bd-a390-66097a8ff3db None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.294885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b86a-f36d-46f0-aa98-8093840c06d9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.319639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7537e71-ed28-48bd-a390-66097a8ff3db None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.330029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddf5f8-0cdd-4527-92c1-7914ca665afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.330639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf5f8-0cdd-4527-92c1-7914ca665afc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.330917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf5f8-0cdd-4527-92c1-7914ca665afc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.331192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf5f8-0cdd-4527-92c1-7914ca665afc None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.331894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddf5f8-0cdd-4527-92c1-7914ca665afc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:07.332195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddf5f8-0cdd-4527-92c1-7914ca665afc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.333569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddf5f8-0cdd-4527-92c1-7914ca665afc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.341913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 550/1071] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:07.342951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff854a7-1940-4b3d-aabf-8cafebd1f75c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.343860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff854a7-1940-4b3d-aabf-8cafebd1f75c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.344174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff854a7-1940-4b3d-aabf-8cafebd1f75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.358939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.359564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.359843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.360145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.360897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:07.361004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0ff854a7-1940-4b3d-aabf-8cafebd1f75c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:07.361204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.362188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0ff854a7-1940-4b3d-aabf-8cafebd1f75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:07.362668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.377526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:07.385389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 522/1072] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:07.386420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.411043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:07.434047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 551/1073] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:07.434047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff854a7-1940-4b3d-aabf-8cafebd1f75c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.497764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28524e88-c39e-4ea5-881a-b41d1bff2f20 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.532630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022bd9c-deb1-48a0-8212-09f6114c4d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.533265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bd9c-deb1-48a0-8212-09f6114c4d90 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.533576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bd9c-deb1-48a0-8212-09f6114c4d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.533850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bd9c-deb1-48a0-8212-09f6114c4d90 None admin] PATH_INFO: `/v3/users/dd06efffc1fb499f84307f61c7ced0ba` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.535015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022bd9c-deb1-48a0-8212-09f6114c4d90 None admin] RBAC: Authorizing `identity:get_user(user_id=dd06efffc1fb499f84307f61c7ced0ba)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.536017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff854a7-1940-4b3d-aabf-8cafebd1f75c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.536437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022bd9c-deb1-48a0-8212-09f6114c4d90 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.538068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 523/1074] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/users/dd06efffc1fb499f84307f61c7ced0ba => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:07.538823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022bd9c-deb1-48a0-8212-09f6114c4d90 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.572199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.572839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.573118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.573685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.584228 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.585689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:25:07.599979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:07.600995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:07.601985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:25:07.601985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:25:07.601985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:07.601985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:07.601985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 552/1075] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:07.601985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.614548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:07.632994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:07.651210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022bd9c-deb1-48a0-8212-09f6114c4d90 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.685256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99333d51-3fda-4b4c-b9c8-a84a0caa37cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.685893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99333d51-3fda-4b4c-b9c8-a84a0caa37cf None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.686175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99333d51-3fda-4b4c-b9c8-a84a0caa37cf None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.686446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99333d51-3fda-4b4c-b9c8-a84a0caa37cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.704797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b591f-b92d-421f-a3db-57dcba81c63c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.738441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143e76d-fc45-4307-88c2-58d4dab36ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.739080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e76d-fc45-4307-88c2-58d4dab36ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.739366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e76d-fc45-4307-88c2-58d4dab36ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.739644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e76d-fc45-4307-88c2-58d4dab36ee2 None admin] PATH_INFO: `/v3/projects/af99cb67bf83485794341607776e18c3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.740809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143e76d-fc45-4307-88c2-58d4dab36ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=af99cb67bf83485794341607776e18c3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.742264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143e76d-fc45-4307-88c2-58d4dab36ee2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.743918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 553/1076] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/projects/af99cb67bf83485794341607776e18c3 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:07.744665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143e76d-fc45-4307-88c2-58d4dab36ee2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.757240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99333d51-3fda-4b4c-b9c8-a84a0caa37cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.758704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99333d51-3fda-4b4c-b9c8-a84a0caa37cf None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.822022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:07.842215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:07.845923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143e76d-fc45-4307-88c2-58d4dab36ee2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.856709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 524/1077] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:07.857533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99333d51-3fda-4b4c-b9c8-a84a0caa37cf None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.881358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.881972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.882249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.882519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.883264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:07.883501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.884847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.895974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 554/1078] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:07.896750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:07.957650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99333d51-3fda-4b4c-b9c8-a84a0caa37cf None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:07.991410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24d109-821e-4968-aead-6521895b4ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:07.992084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24d109-821e-4968-aead-6521895b4ef2 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:07.992370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24d109-821e-4968-aead-6521895b4ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:07.992649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24d109-821e-4968-aead-6521895b4ef2 None admin] PATH_INFO: `/v3/projects/af99cb67bf83485794341607776e18c3/users/dd06efffc1fb499f84307f61c7ced0ba/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:07.994613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24d109-821e-4968-aead-6521895b4ef2 None admin] RBAC: Authorizing `identity:create_grant(project_id=af99cb67bf83485794341607776e18c3, user_id=dd06efffc1fb499f84307f61c7ced0ba, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:07.996044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24d109-821e-4968-aead-6521895b4ef2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:07.997821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550d46a-c31e-4dfd-8e3e-fe23dc2d1795 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:08.011145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 525/1079] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:25:07 2022] PUT /identity/v3/projects/af99cb67bf83485794341607776e18c3/users/dd06efffc1fb499f84307f61c7ced0ba/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:25:08.011940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24d109-821e-4968-aead-6521895b4ef2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:08.034120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:08.035121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:08.036318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:08.036831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:08.053706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:08.056090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:08.057767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:25:08.074370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:08.075643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:08.076686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:25:08.076686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:25:08.076686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:08.076686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:08.078989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 555/1080] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:08.085606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:08.102157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:08.148974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24d109-821e-4968-aead-6521895b4ef2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:08.192936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:08.193935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:08.194406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:08.194897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:08.202483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:08.204540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac325b8-d8dd-4a49-89e0-dcf117453455 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:08.204668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:25:08.217335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:08.218327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:08.219240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 526/1081] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:08.224419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:08.241474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:08.242094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:08.242372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:08.242642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:08.243327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:08.243627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:08.244978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:08.258528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 556/1082] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:08.321134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:08.336665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6909828-6e7d-4ad9-8983-2419ffbe035d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:08.374657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32ee48-1385-439d-8d64-0fbcf68190d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:08.375338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32ee48-1385-439d-8d64-0fbcf68190d1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:08.375624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32ee48-1385-439d-8d64-0fbcf68190d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:08.375901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32ee48-1385-439d-8d64-0fbcf68190d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:08.376667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32ee48-1385-439d-8d64-0fbcf68190d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:08.376977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32ee48-1385-439d-8d64-0fbcf68190d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:08.378484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32ee48-1385-439d-8d64-0fbcf68190d1 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:08.391942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 527/1083] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:08.397980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:08.478955 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:08.514429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a248ff-b309-47db-b7b6-5d9bfc67dddf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:08.550529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4852d6d-afcb-4d1c-a372-4de524c99d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:08.551212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4852d6d-afcb-4d1c-a372-4de524c99d71 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:08.551490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4852d6d-afcb-4d1c-a372-4de524c99d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:08.551770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4852d6d-afcb-4d1c-a372-4de524c99d71 None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/9f558687eefe4a26beb7d69151221660/roles/030852a56a8145e3b8b7c785b4ace178` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:08.553951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4852d6d-afcb-4d1c-a372-4de524c99d71 None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=9f558687eefe4a26beb7d69151221660, role_id=030852a56a8145e3b8b7c785b4ace178)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:08.555381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4852d6d-afcb-4d1c-a372-4de524c99d71 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:08.569555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 557/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:25:08 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/9f558687eefe4a26beb7d69151221660/roles/030852a56a8145e3b8b7c785b4ace178 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:25:10.067641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b95437-2d05-4f26-9965-bfd4dcd8d38c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.068176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b95437-2d05-4f26-9965-bfd4dcd8d38c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.068176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b95437-2d05-4f26-9965-bfd4dcd8d38c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.069584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 528/1085] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:10.075053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2022f-2544-475d-8ea9-48213ac69ce8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.075466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2022f-2544-475d-8ea9-48213ac69ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.075606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2022f-2544-475d-8ea9-48213ac69ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.094444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a7f2022f-2544-475d-8ea9-48213ac69ce8 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:10.095618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a7f2022f-2544-475d-8ea9-48213ac69ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:10.166801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 558/1086] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:10.191720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b95437-2d05-4f26-9965-bfd4dcd8d38c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:10.293185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b95437-2d05-4f26-9965-bfd4dcd8d38c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:10.326719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa8f88-fafb-4358-a76b-50dd8aa06f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:10.327289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa8f88-fafb-4358-a76b-50dd8aa06f71 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.327662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa8f88-fafb-4358-a76b-50dd8aa06f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.327944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa8f88-fafb-4358-a76b-50dd8aa06f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.337884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa8f88-fafb-4358-a76b-50dd8aa06f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-46aa8f88-fafb-4358-a76b-50dd8aa06f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:10.342832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:10.344734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:10.345671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 529/1087] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:10.350872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2022f-2544-475d-8ea9-48213ac69ce8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:10.410498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:10.451013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2022f-2544-475d-8ea9-48213ac69ce8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:10.485446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc5092-9350-49ad-959b-087ac4b17dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:10.486067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5092-9350-49ad-959b-087ac4b17dfe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.486346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5092-9350-49ad-959b-087ac4b17dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.486617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc5092-9350-49ad-959b-087ac4b17dfe None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.487331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc5092-9350-49ad-959b-087ac4b17dfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:10.487638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc5092-9350-49ad-959b-087ac4b17dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:10.488987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc5092-9350-49ad-959b-087ac4b17dfe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:10.496997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 559/1088] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:10.502665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa8f88-fafb-4358-a76b-50dd8aa06f71 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:10.513802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:10.580054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf5421-cf34-41f9-9505-1a61b9ad6896 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.580261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf5421-cf34-41f9-9505-1a61b9ad6896 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.580536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf5421-cf34-41f9-9505-1a61b9ad6896 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.582075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 560/1089] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:25:10.586712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479f1f-6e60-41d0-a760-959522547aff None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.586999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479f1f-6e60-41d0-a760-959522547aff None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.587280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9479f1f-6e60-41d0-a760-959522547aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.603822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa8f88-fafb-4358-a76b-50dd8aa06f71 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:10.606070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d9479f1f-6e60-41d0-a760-959522547aff None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:10.607261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d9479f1f-6e60-41d0-a760-959522547aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:10.638762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1948f4-db84-41f6-bcc9-e8c5d94682da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:10.639396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1948f4-db84-41f6-bcc9-e8c5d94682da None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.639675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1948f4-db84-41f6-bcc9-e8c5d94682da None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.640447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1948f4-db84-41f6-bcc9-e8c5d94682da None admin] PATH_INFO: `/v3/users/dd06efffc1fb499f84307f61c7ced0ba` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.641685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1948f4-db84-41f6-bcc9-e8c5d94682da None admin] RBAC: Authorizing `identity:get_user(user_id=dd06efffc1fb499f84307f61c7ced0ba)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:10.643088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1948f4-db84-41f6-bcc9-e8c5d94682da None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:10.644680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 530/1090] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:10 2022] GET /identity/v3/users/dd06efffc1fb499f84307f61c7ced0ba => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:10.650273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1948f4-db84-41f6-bcc9-e8c5d94682da None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:10.676393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 561/1091] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:25:10.699839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9479f1f-6e60-41d0-a760-959522547aff None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:10.727894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:10.750781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1948f4-db84-41f6-bcc9-e8c5d94682da None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:10.776987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:10.804295 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e94778-4424-45a1-8845-d00e08b51439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:10.804929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e94778-4424-45a1-8845-d00e08b51439 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.805216 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e94778-4424-45a1-8845-d00e08b51439 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.805525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e94778-4424-45a1-8845-d00e08b51439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.828742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9479f1f-6e60-41d0-a760-959522547aff None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:10.865754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfed643-0f7c-4a57-80a5-e4e317eee0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:10.866458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfed643-0f7c-4a57-80a5-e4e317eee0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:10.866740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfed643-0f7c-4a57-80a5-e4e317eee0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:10.867015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfed643-0f7c-4a57-80a5-e4e317eee0b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:10.878140 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfed643-0f7c-4a57-80a5-e4e317eee0b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:10.881993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e94778-4424-45a1-8845-d00e08b51439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6dfed643-0f7c-4a57-80a5-e4e317eee0b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:10.883039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:10.884044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 06:25:10.884935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:10.885630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e94778-4424-45a1-8845-d00e08b51439 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:10.885719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 562/1092] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Jul 28 06:25:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:25:10.890905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfed643-0f7c-4a57-80a5-e4e317eee0b2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:10.965263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:10.991819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 531/1093] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:10.998307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e94778-4424-45a1-8845-d00e08b51439 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:11.004343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfed643-0f7c-4a57-80a5-e4e317eee0b2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:11.022401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:11.043210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:11.043831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:11.044122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:11.044399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:11.045111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:11.045922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:11.047398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:11.055398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 563/1094] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jul 28 06:25:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:11.061071 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:11.107403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e94778-4424-45a1-8845-d00e08b51439 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:11.143199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4545-1bf3-4ad5-9885-89a756659bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:11.143760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4545-1bf3-4ad5-9885-89a756659bbc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:11.144043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4545-1bf3-4ad5-9885-89a756659bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:11.144326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4545-1bf3-4ad5-9885-89a756659bbc None admin] PATH_INFO: `/v3/projects/af99cb67bf83485794341607776e18c3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:11.145525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c4545-1bf3-4ad5-9885-89a756659bbc None admin] RBAC: Authorizing `identity:get_project(project_id=af99cb67bf83485794341607776e18c3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:11.146942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c4545-1bf3-4ad5-9885-89a756659bbc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:11.148794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 532/1095] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:10 2022] GET /identity/v3/projects/af99cb67bf83485794341607776e18c3 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:11.154587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4545-1bf3-4ad5-9885-89a756659bbc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:11.173060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cef750-0ee4-4ec3-89e4-92e77c8b2efd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:11.200793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:11.208398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:11.209038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:11.209345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:11.209623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:11.220157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:11.221632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:25:11.235760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:11.236778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:11.237902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:25:11.237902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:25:11.237902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:11.237902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:11.237902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 564/1096] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:25:11.242540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:11.257738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4545-1bf3-4ad5-9885-89a756659bbc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:11.270933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:11.291824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:11.292434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:11.292719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:11.292994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:11.293808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=af99cb67bf83485794341607776e18c3, user.id=dd06efffc1fb499f84307f61c7ced0ba) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:11.294120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:11.295536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:11.302007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 533/1097] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:11 2022] GET /identity/v3/role_assignments?user.id=dd06efffc1fb499f84307f61c7ced0ba&scope.project.id=af99cb67bf83485794341607776e18c3&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:11.350417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daad19-f645-42cd-8dda-ec7c0c109294 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:11.387973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:11.388602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:11.388884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:11.389159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:11.390499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:11.391432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:11.392838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:11.407787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 565/1098] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:11.413494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:11.519930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578938-07d0-4d06-9deb-3da9a680cad0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:11.528770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:11.555165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:11.555800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:11.556081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:11.556354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:11.564149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:11.565617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:25:11.576903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:11.578690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:11.579652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 534/1099] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Jul 28 06:25:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:25:11.584858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:11.674979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:11.688046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7b103-4159-4827-a8a1-360580d543b1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:11.723084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:11.723681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:11.723964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:11.724235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:11.725016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:11.725344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:11.726749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:11.743678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 566/1100] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Jul 28 06:25:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:11.749855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:11.848454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a690c9c-3434-427b-925b-e016de7c781a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:11.881387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37a92f-62cf-49c5-a210-13cc3efdb828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:11.882001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a92f-62cf-49c5-a210-13cc3efdb828 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:11.882282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a92f-62cf-49c5-a210-13cc3efdb828 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:11.882552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a92f-62cf-49c5-a210-13cc3efdb828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:11.917280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:11.950589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37a92f-62cf-49c5-a210-13cc3efdb828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:11.951968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37a92f-62cf-49c5-a210-13cc3efdb828 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:12.049357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 535/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 06:25:12.055068 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:12.156888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8c000-561d-4645-85d2-cfb2c4596903 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:12.192529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:12.193436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:12.193721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:12.193995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:12.204353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:12.206080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:12.212691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:25:12.221682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:12.222869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:12.223968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:25:12.223968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:25:12.223968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:25:12.223968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:12.225441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 567/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:12.228442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37a92f-62cf-49c5-a210-13cc3efdb828 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:12.293011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:12.327266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37a92f-62cf-49c5-a210-13cc3efdb828 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:12.361021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:12.361665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:12.361950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:12.362227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:12.362909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:12.363213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:12.364581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:12.376218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 536/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:12.381886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:12.483481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ab08b-f7d2-4b27-9331-07ed8905ca20 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:12.516941 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:12.517612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:12.517897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:12.518175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:12.524677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:12.526129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:25:12.537870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:12.538918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:12.545057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 568/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:12.545700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:12.646167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3d08d-6ab5-4727-84c5-d750e74d13ea None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:12.672830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:12.681851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ad84c-42a2-4f1f-a74b-e6b616cc3088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:12.682473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ad84c-42a2-4f1f-a74b-e6b616cc3088 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:12.682758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ad84c-42a2-4f1f-a74b-e6b616cc3088 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:12.683033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ad84c-42a2-4f1f-a74b-e6b616cc3088 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:12.683784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ad84c-42a2-4f1f-a74b-e6b616cc3088 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:12.684093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ad84c-42a2-4f1f-a74b-e6b616cc3088 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:12.685541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ad84c-42a2-4f1f-a74b-e6b616cc3088 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:12.697944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 537/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:12.705052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:12.763000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:12.804572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448aac3-d5f1-4402-b495-1a7c96beb44a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:12.838090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc03a6-26ca-47c4-9c4b-71549a396ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:12.838704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc03a6-26ca-47c4-9c4b-71549a396ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:12.838987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc03a6-26ca-47c4-9c4b-71549a396ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:12.839262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc03a6-26ca-47c4-9c4b-71549a396ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:12.840043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc03a6-26ca-47c4-9c4b-71549a396ab1 None admin] RBAC: Adding query filter params (role.id=030852a56a8145e3b8b7c785b4ace178, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=9f558687eefe4a26beb7d69151221660) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:12.840353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc03a6-26ca-47c4-9c4b-71549a396ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:12.841812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc03a6-26ca-47c4-9c4b-71549a396ab1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:12.847750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 569/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:12 2022] GET /identity/v3/role_assignments?user.id=9f558687eefe4a26beb7d69151221660&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=030852a56a8145e3b8b7c785b4ace178 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:13.345973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc848c-8d3b-49c0-b47a-64a5c153e9d5 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:13.346492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc848c-8d3b-49c0-b47a-64a5c153e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:13.346606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc848c-8d3b-49c0-b47a-64a5c153e9d5 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:13.347867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 538/1107] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:13.353272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440c0cf-c049-43a3-ab56-7f70fb2060bb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:13.353737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440c0cf-c049-43a3-ab56-7f70fb2060bb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:13.353893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440c0cf-c049-43a3-ab56-7f70fb2060bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:13.372778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9440c0cf-c049-43a3-ab56-7f70fb2060bb None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:13.374023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9440c0cf-c049-43a3-ab56-7f70fb2060bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:13.447199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 570/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:13.473105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc848c-8d3b-49c0-b47a-64a5c153e9d5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:13.579114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc848c-8d3b-49c0-b47a-64a5c153e9d5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:13.613267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d920164-7614-4e86-b621-24e2f48ba51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:13.613871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d920164-7614-4e86-b621-24e2f48ba51d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:13.614155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d920164-7614-4e86-b621-24e2f48ba51d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:13.614431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d920164-7614-4e86-b621-24e2f48ba51d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:13.615587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d920164-7614-4e86-b621-24e2f48ba51d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:13.617014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d920164-7614-4e86-b621-24e2f48ba51d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:13.618729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 539/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:13.625162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440c0cf-c049-43a3-ab56-7f70fb2060bb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:13.679857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:13.728959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440c0cf-c049-43a3-ab56-7f70fb2060bb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:13.762968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304ce17-78ae-48a2-b95c-2dfe7dc2000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:13.763578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304ce17-78ae-48a2-b95c-2dfe7dc2000f None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:13.763862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304ce17-78ae-48a2-b95c-2dfe7dc2000f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:13.764139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304ce17-78ae-48a2-b95c-2dfe7dc2000f None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:13.764847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304ce17-78ae-48a2-b95c-2dfe7dc2000f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:13.766336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304ce17-78ae-48a2-b95c-2dfe7dc2000f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:13.789695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 571/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:25:13 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:25:15.871286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92952a4d-7238-4af3-80a7-0f123ca94abc None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:15.871871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92952a4d-7238-4af3-80a7-0f123ca94abc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:15.871871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92952a4d-7238-4af3-80a7-0f123ca94abc None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:15.873173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 540/1111] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:15.878984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3b82-aa55-436c-8f0d-a5c97eb260a3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:15.879395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3b82-aa55-436c-8f0d-a5c97eb260a3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:15.879545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3b82-aa55-436c-8f0d-a5c97eb260a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:15.897721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4d3e3b82-aa55-436c-8f0d-a5c97eb260a3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:15.898900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4d3e3b82-aa55-436c-8f0d-a5c97eb260a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:15.963102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 572/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:15.986208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92952a4d-7238-4af3-80a7-0f123ca94abc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:16.041556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:16.084756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92952a4d-7238-4af3-80a7-0f123ca94abc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:16.117312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3b515-57fc-482e-a0e1-b10c26a0c7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:16.117940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3b515-57fc-482e-a0e1-b10c26a0c7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:16.118217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3b515-57fc-482e-a0e1-b10c26a0c7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:16.118486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3b515-57fc-482e-a0e1-b10c26a0c7b2 None admin] PATH_INFO: `/v3/roles/74f371799fd245bf987e798f5c342e6a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:16.119895 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3b515-57fc-482e-a0e1-b10c26a0c7b2 None admin] RBAC: Authorizing `identity:get_role(role_id=74f371799fd245bf987e798f5c342e6a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:16.121255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3b515-57fc-482e-a0e1-b10c26a0c7b2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:16.122639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 541/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:15 2022] GET /identity/v3/roles/74f371799fd245bf987e798f5c342e6a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:16.129000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e3b82-aa55-436c-8f0d-a5c97eb260a3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:16.167481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:16.228782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e3b82-aa55-436c-8f0d-a5c97eb260a3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:16.262991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e53003-22f5-4404-9fe5-2ac989f46d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:16.263631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e53003-22f5-4404-9fe5-2ac989f46d07 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:16.263914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e53003-22f5-4404-9fe5-2ac989f46d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:16.264190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e53003-22f5-4404-9fe5-2ac989f46d07 None admin] PATH_INFO: `/v3/users/6ec4e7bc1bf44ef49b6a8eb35b881874` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:16.277131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e53003-22f5-4404-9fe5-2ac989f46d07 None admin] RBAC: Authorizing `identity:get_user(user_id=6ec4e7bc1bf44ef49b6a8eb35b881874)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:16.278591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e53003-22f5-4404-9fe5-2ac989f46d07 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:16.280205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 573/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:16 2022] GET /identity/v3/users/6ec4e7bc1bf44ef49b6a8eb35b881874 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:16.286702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3b515-57fc-482e-a0e1-b10c26a0c7b2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:16.385062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3b515-57fc-482e-a0e1-b10c26a0c7b2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:16.408023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:16.418664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fd010-4828-4078-a2fc-74c7834c076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:16.419281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd010-4828-4078-a2fc-74c7834c076a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:16.419566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd010-4828-4078-a2fc-74c7834c076a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:16.419841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd010-4828-4078-a2fc-74c7834c076a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:16.486766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fd010-4828-4078-a2fc-74c7834c076a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:16.488193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fd010-4828-4078-a2fc-74c7834c076a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:16.584042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 542/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:16.589916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e53003-22f5-4404-9fe5-2ac989f46d07 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:16.682475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:16.689162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e53003-22f5-4404-9fe5-2ac989f46d07 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:16.724050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1f5e8-93fb-43bd-817c-5fb55f0b0488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:16.724679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1f5e8-93fb-43bd-817c-5fb55f0b0488 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:16.724960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1f5e8-93fb-43bd-817c-5fb55f0b0488 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:16.725241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1f5e8-93fb-43bd-817c-5fb55f0b0488 None admin] PATH_INFO: `/v3/projects/af99cb67bf83485794341607776e18c3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:16.726410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1f5e8-93fb-43bd-817c-5fb55f0b0488 None admin] RBAC: Authorizing `identity:get_project(project_id=af99cb67bf83485794341607776e18c3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:16.727840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1f5e8-93fb-43bd-817c-5fb55f0b0488 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:16.729541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 574/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:16 2022] GET /identity/v3/projects/af99cb67bf83485794341607776e18c3 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:16.735627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fd010-4828-4078-a2fc-74c7834c076a None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:16.759278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:16.832778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fd010-4828-4078-a2fc-74c7834c076a None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:16.865809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743fe3-e294-4c87-ad25-af37db477b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:16.866411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743fe3-e294-4c87-ad25-af37db477b43 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:16.866696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743fe3-e294-4c87-ad25-af37db477b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:16.866975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743fe3-e294-4c87-ad25-af37db477b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:16.867752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d743fe3-e294-4c87-ad25-af37db477b43 None admin] RBAC: Adding query filter params (role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=af99cb67bf83485794341607776e18c3, user.id=6ec4e7bc1bf44ef49b6a8eb35b881874) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:16.868063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d743fe3-e294-4c87-ad25-af37db477b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:16.869498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d743fe3-e294-4c87-ad25-af37db477b43 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:16.874733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 543/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:16 2022] GET /identity/v3/role_assignments?user.id=6ec4e7bc1bf44ef49b6a8eb35b881874&scope.project.id=af99cb67bf83485794341607776e18c3&role.id=74f371799fd245bf987e798f5c342e6a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:18.852467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed1ac-5d39-4dab-8a4c-6d0e3370ac99 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:18.853036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed1ac-5d39-4dab-8a4c-6d0e3370ac99 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:18.853036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed1ac-5d39-4dab-8a4c-6d0e3370ac99 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:18.854407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 575/1118] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:18.859603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88a80-28bc-4b5f-97d6-56ea0eed65f9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:18.860018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88a80-28bc-4b5f-97d6-56ea0eed65f9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:18.860156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88a80-28bc-4b5f-97d6-56ea0eed65f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:18.878457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0bd88a80-28bc-4b5f-97d6-56ea0eed65f9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:18.879652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0bd88a80-28bc-4b5f-97d6-56ea0eed65f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:18.945530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 544/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:18.968294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed1ac-5d39-4dab-8a4c-6d0e3370ac99 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:19.067621 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed1ac-5d39-4dab-8a4c-6d0e3370ac99 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:19.101236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7afb-e367-4835-9e3f-2850928688b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:19.101899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7afb-e367-4835-9e3f-2850928688b9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:19.102179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7afb-e367-4835-9e3f-2850928688b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:19.102452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad7afb-e367-4835-9e3f-2850928688b9 None admin] PATH_INFO: `/v3/roles/74f371799fd245bf987e798f5c342e6a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:19.103932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad7afb-e367-4835-9e3f-2850928688b9 None admin] RBAC: Authorizing `identity:get_role(role_id=74f371799fd245bf987e798f5c342e6a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:19.105374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad7afb-e367-4835-9e3f-2850928688b9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:19.106702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 576/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:18 2022] GET /identity/v3/roles/74f371799fd245bf987e798f5c342e6a => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:19.112902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd88a80-28bc-4b5f-97d6-56ea0eed65f9 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:19.131154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:19.208202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd88a80-28bc-4b5f-97d6-56ea0eed65f9 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:19.240514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451afbcf-8899-4c1d-8212-4859ee1abcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:19.241143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451afbcf-8899-4c1d-8212-4859ee1abcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:19.241452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451afbcf-8899-4c1d-8212-4859ee1abcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:19.241726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451afbcf-8899-4c1d-8212-4859ee1abcc2 None admin] PATH_INFO: `/v3/users/6ec4e7bc1bf44ef49b6a8eb35b881874` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:19.242865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451afbcf-8899-4c1d-8212-4859ee1abcc2 None admin] RBAC: Authorizing `identity:get_user(user_id=6ec4e7bc1bf44ef49b6a8eb35b881874)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:19.244277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451afbcf-8899-4c1d-8212-4859ee1abcc2 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:19.245867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 545/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:19 2022] GET /identity/v3/users/6ec4e7bc1bf44ef49b6a8eb35b881874 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:19.252446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7afb-e367-4835-9e3f-2850928688b9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:19.283054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:19.350048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad7afb-e367-4835-9e3f-2850928688b9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:19.382796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cee460-4cfa-4c67-9bc9-1a722e08f022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:19.383360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cee460-4cfa-4c67-9bc9-1a722e08f022 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:19.383643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cee460-4cfa-4c67-9bc9-1a722e08f022 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:19.383918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cee460-4cfa-4c67-9bc9-1a722e08f022 None admin] PATH_INFO: `/v3/projects/af99cb67bf83485794341607776e18c3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:19.385062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cee460-4cfa-4c67-9bc9-1a722e08f022 None admin] RBAC: Authorizing `identity:get_project(project_id=af99cb67bf83485794341607776e18c3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:19.386507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cee460-4cfa-4c67-9bc9-1a722e08f022 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:19.388206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 577/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:19 2022] GET /identity/v3/projects/af99cb67bf83485794341607776e18c3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:19.394621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451afbcf-8899-4c1d-8212-4859ee1abcc2 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:19.476864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:19.491568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451afbcf-8899-4c1d-8212-4859ee1abcc2 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:19.525637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22735e7-ad07-48b6-9e20-2e60e5959a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:19.526287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22735e7-ad07-48b6-9e20-2e60e5959a1e None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:19.526566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22735e7-ad07-48b6-9e20-2e60e5959a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:19.526838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22735e7-ad07-48b6-9e20-2e60e5959a1e None admin] PATH_INFO: `/v3/projects/af99cb67bf83485794341607776e18c3/users/6ec4e7bc1bf44ef49b6a8eb35b881874/roles/74f371799fd245bf987e798f5c342e6a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:19.528966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22735e7-ad07-48b6-9e20-2e60e5959a1e None admin] RBAC: Authorizing `identity:create_grant(project_id=af99cb67bf83485794341607776e18c3, user_id=6ec4e7bc1bf44ef49b6a8eb35b881874, role_id=74f371799fd245bf987e798f5c342e6a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:19.530542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22735e7-ad07-48b6-9e20-2e60e5959a1e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:19.542793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 546/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:25:19 2022] PUT /identity/v3/projects/af99cb67bf83485794341607776e18c3/users/6ec4e7bc1bf44ef49b6a8eb35b881874/roles/74f371799fd245bf987e798f5c342e6a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:25:21.559332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c93fe-65ec-40ec-9f3e-ef6f2dfe2bf5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:21.559874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c93fe-65ec-40ec-9f3e-ef6f2dfe2bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:21.559874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c93fe-65ec-40ec-9f3e-ef6f2dfe2bf5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:21.561222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 578/1124] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:21.566782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc7fb2-65da-428e-995b-0ca6a56c02d5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:21.567154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc7fb2-65da-428e-995b-0ca6a56c02d5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:21.567331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc7fb2-65da-428e-995b-0ca6a56c02d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:21.585414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-17fc7fb2-65da-428e-995b-0ca6a56c02d5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:21.586581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-17fc7fb2-65da-428e-995b-0ca6a56c02d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:21.651626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 547/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:21.675669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c93fe-65ec-40ec-9f3e-ef6f2dfe2bf5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:21.777595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:21.782451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c93fe-65ec-40ec-9f3e-ef6f2dfe2bf5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:21.816778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d819-3fed-428f-b359-e8c557548cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:21.817435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d819-3fed-428f-b359-e8c557548cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:21.817715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d819-3fed-428f-b359-e8c557548cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:21.817987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d819-3fed-428f-b359-e8c557548cc5 None admin] PATH_INFO: `/v3/roles/74f371799fd245bf987e798f5c342e6a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:21.819396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7d819-3fed-428f-b359-e8c557548cc5 None admin] RBAC: Authorizing `identity:get_role(role_id=74f371799fd245bf987e798f5c342e6a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:21.820748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7d819-3fed-428f-b359-e8c557548cc5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:21.822104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 579/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:21 2022] GET /identity/v3/roles/74f371799fd245bf987e798f5c342e6a => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:21.828491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc7fb2-65da-428e-995b-0ca6a56c02d5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:21.902640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:21.926670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc7fb2-65da-428e-995b-0ca6a56c02d5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:21.959974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c6791-3015-4386-941a-d7631eac757c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:21.960607 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c6791-3015-4386-941a-d7631eac757c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:21.960891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c6791-3015-4386-941a-d7631eac757c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:21.961169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c6791-3015-4386-941a-d7631eac757c None admin] PATH_INFO: `/v3/users/6ec4e7bc1bf44ef49b6a8eb35b881874` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:21.962356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c6791-3015-4386-941a-d7631eac757c None admin] RBAC: Authorizing `identity:get_user(user_id=6ec4e7bc1bf44ef49b6a8eb35b881874)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:21.963773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c6791-3015-4386-941a-d7631eac757c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:21.965382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 548/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:21 2022] GET /identity/v3/users/6ec4e7bc1bf44ef49b6a8eb35b881874 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:21.971260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d819-3fed-428f-b359-e8c557548cc5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:22.067780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d819-3fed-428f-b359-e8c557548cc5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:22.100227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3dc4a-75df-4f27-813d-451712ee2b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:22.100838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3dc4a-75df-4f27-813d-451712ee2b4b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:22.101120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3dc4a-75df-4f27-813d-451712ee2b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:22.101427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3dc4a-75df-4f27-813d-451712ee2b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:22.135012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:22.166679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3dc4a-75df-4f27-813d-451712ee2b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:22.168111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3dc4a-75df-4f27-813d-451712ee2b4b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:22.265070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 580/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:22.270999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c6791-3015-4386-941a-d7631eac757c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:22.368670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c6791-3015-4386-941a-d7631eac757c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:22.400905 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40cc6a-0ca3-4a64-a061-3c261034e776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:22.401564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40cc6a-0ca3-4a64-a061-3c261034e776 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:22.401846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40cc6a-0ca3-4a64-a061-3c261034e776 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:22.402119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40cc6a-0ca3-4a64-a061-3c261034e776 None admin] PATH_INFO: `/v3/projects/af99cb67bf83485794341607776e18c3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:22.403256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40cc6a-0ca3-4a64-a061-3c261034e776 None admin] RBAC: Authorizing `identity:get_project(project_id=af99cb67bf83485794341607776e18c3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:22.404671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40cc6a-0ca3-4a64-a061-3c261034e776 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:22.406965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 549/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:22 2022] GET /identity/v3/projects/af99cb67bf83485794341607776e18c3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:22.413008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3dc4a-75df-4f27-813d-451712ee2b4b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:22.478820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:22.512219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3dc4a-75df-4f27-813d-451712ee2b4b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:22.545419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8ed42-4ff5-4fae-b885-b6912d437294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:22.546633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8ed42-4ff5-4fae-b885-b6912d437294 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:22.546914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8ed42-4ff5-4fae-b885-b6912d437294 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:22.547192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8ed42-4ff5-4fae-b885-b6912d437294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:22.547972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8ed42-4ff5-4fae-b885-b6912d437294 None admin] RBAC: Adding query filter params (role.id=74f371799fd245bf987e798f5c342e6a, scope.project.id=af99cb67bf83485794341607776e18c3, user.id=6ec4e7bc1bf44ef49b6a8eb35b881874) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:22.548284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8ed42-4ff5-4fae-b885-b6912d437294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:22.549734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8ed42-4ff5-4fae-b885-b6912d437294 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:22.555337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 581/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:22 2022] GET /identity/v3/role_assignments?user.id=6ec4e7bc1bf44ef49b6a8eb35b881874&scope.project.id=af99cb67bf83485794341607776e18c3&role.id=74f371799fd245bf987e798f5c342e6a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:24.549197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f22fb-5907-4577-87eb-0394b5bab244 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:24.549764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f22fb-5907-4577-87eb-0394b5bab244 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:24.549764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f22fb-5907-4577-87eb-0394b5bab244 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:24.551116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 550/1131] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:24.556420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28968477-4ed2-4ff0-82d1-bdc7f157a3cc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:24.556808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28968477-4ed2-4ff0-82d1-bdc7f157a3cc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:24.556977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28968477-4ed2-4ff0-82d1-bdc7f157a3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:24.576103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-28968477-4ed2-4ff0-82d1-bdc7f157a3cc None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:24.577881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-28968477-4ed2-4ff0-82d1-bdc7f157a3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:24.641377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 582/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:24.664559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f22fb-5907-4577-87eb-0394b5bab244 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:24.679638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:24.764597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f22fb-5907-4577-87eb-0394b5bab244 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:24.797847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd99b9-9344-4ff8-8fe2-a6e7256082cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:24.798481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd99b9-9344-4ff8-8fe2-a6e7256082cb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:24.798768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd99b9-9344-4ff8-8fe2-a6e7256082cb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:24.799053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd99b9-9344-4ff8-8fe2-a6e7256082cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:24.800203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd99b9-9344-4ff8-8fe2-a6e7256082cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:24.801661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd99b9-9344-4ff8-8fe2-a6e7256082cb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:24.803242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 551/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:24.809010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28968477-4ed2-4ff0-82d1-bdc7f157a3cc None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:24.870452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:24.910114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28968477-4ed2-4ff0-82d1-bdc7f157a3cc None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:24.943402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64947ccc-c6fd-4a68-85d7-ecd102ce46da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:24.944010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64947ccc-c6fd-4a68-85d7-ecd102ce46da None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:24.944298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64947ccc-c6fd-4a68-85d7-ecd102ce46da None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:24.944571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64947ccc-c6fd-4a68-85d7-ecd102ce46da None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:24.945254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64947ccc-c6fd-4a68-85d7-ecd102ce46da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:24.946699 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64947ccc-c6fd-4a68-85d7-ecd102ce46da None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:24.973171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 583/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:25:24 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:25:27.006001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f078e95-0ec4-42dc-946a-abc6a6ce1192 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:27.006566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f078e95-0ec4-42dc-946a-abc6a6ce1192 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:27.006566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f078e95-0ec4-42dc-946a-abc6a6ce1192 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:27.007911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 552/1135] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:27.013065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc1447-a8fc-411e-a368-6b081cd0552d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:27.013485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc1447-a8fc-411e-a368-6b081cd0552d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:27.013670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc1447-a8fc-411e-a368-6b081cd0552d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:27.032371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d7dc1447-a8fc-411e-a368-6b081cd0552d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:27.033592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d7dc1447-a8fc-411e-a368-6b081cd0552d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:27.099672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 584/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:27.124691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f078e95-0ec4-42dc-946a-abc6a6ce1192 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:27.207048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:27.227318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f078e95-0ec4-42dc-946a-abc6a6ce1192 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:27.261577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df082880-715c-4395-aca6-1a6c86a77f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:27.262141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df082880-715c-4395-aca6-1a6c86a77f05 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:27.262423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df082880-715c-4395-aca6-1a6c86a77f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:27.262697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df082880-715c-4395-aca6-1a6c86a77f05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:27.263837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df082880-715c-4395-aca6-1a6c86a77f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:27.265745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df082880-715c-4395-aca6-1a6c86a77f05 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:27.267316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 553/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:25:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:27.273027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc1447-a8fc-411e-a368-6b081cd0552d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:27.317106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:27.369648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc1447-a8fc-411e-a368-6b081cd0552d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:27.402816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bd3ac-85a4-4cc7-bead-c4882e87ebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:27.403429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bd3ac-85a4-4cc7-bead-c4882e87ebb9 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:27.403710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bd3ac-85a4-4cc7-bead-c4882e87ebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:27.403982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bd3ac-85a4-4cc7-bead-c4882e87ebb9 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:27.404653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bd3ac-85a4-4cc7-bead-c4882e87ebb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:27.406551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bd3ac-85a4-4cc7-bead-c4882e87ebb9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:27.427500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 585/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:25:27 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:25:29.500793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebf58b-9613-4a5f-a63f-dccc6290ad61 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:29.501333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebf58b-9613-4a5f-a63f-dccc6290ad61 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:29.501441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebf58b-9613-4a5f-a63f-dccc6290ad61 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:29.502708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 554/1139] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:29.507867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff007d6-16a4-4fae-97aa-9bcdc041b708 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:29.508253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff007d6-16a4-4fae-97aa-9bcdc041b708 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:29.508432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff007d6-16a4-4fae-97aa-9bcdc041b708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:29.526652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5ff007d6-16a4-4fae-97aa-9bcdc041b708 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:29.527837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5ff007d6-16a4-4fae-97aa-9bcdc041b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:29.595480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 586/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:29.618991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebf58b-9613-4a5f-a63f-dccc6290ad61 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:29.718316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebf58b-9613-4a5f-a63f-dccc6290ad61 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:29.752738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d457fa0-75a1-4a5d-928a-aa56cfec797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:29.753386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d457fa0-75a1-4a5d-928a-aa56cfec797b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:29.753669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d457fa0-75a1-4a5d-928a-aa56cfec797b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:29.753945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d457fa0-75a1-4a5d-928a-aa56cfec797b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:29.763755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d457fa0-75a1-4a5d-928a-aa56cfec797b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-0d457fa0-75a1-4a5d-928a-aa56cfec797b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:29.769074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:29.770315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:29.771417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 555/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:29.776581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff007d6-16a4-4fae-97aa-9bcdc041b708 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:29.808342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:29.878853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff007d6-16a4-4fae-97aa-9bcdc041b708 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:29.911995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:29.912619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:29.912903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:29.913178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:29.913922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:29.914229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:29.915592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:29.924017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 587/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:29.929853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d457fa0-75a1-4a5d-928a-aa56cfec797b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:29.941002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:30.028170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d457fa0-75a1-4a5d-928a-aa56cfec797b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:30.061481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931b49c-c42a-4014-b0eb-8ad6eb2cce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:30.062113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931b49c-c42a-4014-b0eb-8ad6eb2cce4c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:30.062395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931b49c-c42a-4014-b0eb-8ad6eb2cce4c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:30.062666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931b49c-c42a-4014-b0eb-8ad6eb2cce4c None admin] PATH_INFO: `/v3/users/b473d853ce2b4b828680acac501cc02d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:30.075539 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931b49c-c42a-4014-b0eb-8ad6eb2cce4c None admin] RBAC: Authorizing `identity:get_user(user_id=b473d853ce2b4b828680acac501cc02d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:30.077125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931b49c-c42a-4014-b0eb-8ad6eb2cce4c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:30.078860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 556/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:29 2022] GET /identity/v3/users/b473d853ce2b4b828680acac501cc02d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:30.084532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:30.174935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:30.183584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec05d95-9ce7-4776-a07b-36a7ad28b433 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:30.216246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc837f-bf8f-4c24-8ef3-00128f1c88cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:30.216853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc837f-bf8f-4c24-8ef3-00128f1c88cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:30.217129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc837f-bf8f-4c24-8ef3-00128f1c88cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:30.217993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc837f-bf8f-4c24-8ef3-00128f1c88cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:30.283117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc837f-bf8f-4c24-8ef3-00128f1c88cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:30.284473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc837f-bf8f-4c24-8ef3-00128f1c88cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:30.371275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:30.381105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 588/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:30.387175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931b49c-c42a-4014-b0eb-8ad6eb2cce4c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:30.451201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:30.486416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931b49c-c42a-4014-b0eb-8ad6eb2cce4c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:30.520183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc414f98-a40c-41c4-8b39-130ca9262bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:30.520817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc414f98-a40c-41c4-8b39-130ca9262bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:30.521100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc414f98-a40c-41c4-8b39-130ca9262bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:30.521405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc414f98-a40c-41c4-8b39-130ca9262bd0 None admin] PATH_INFO: `/v3/projects/f9e5c137ffca43c1bbba55cc943e7814` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:30.522553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc414f98-a40c-41c4-8b39-130ca9262bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=f9e5c137ffca43c1bbba55cc943e7814)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:30.523979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc414f98-a40c-41c4-8b39-130ca9262bd0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:30.525659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 557/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:30 2022] GET /identity/v3/projects/f9e5c137ffca43c1bbba55cc943e7814 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:25:30.531921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc837f-bf8f-4c24-8ef3-00128f1c88cf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:30.629117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc837f-bf8f-4c24-8ef3-00128f1c88cf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:30.661838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50818c0-cfff-4c33-a5e4-1959410d4071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:30.662446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50818c0-cfff-4c33-a5e4-1959410d4071 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:30.662723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50818c0-cfff-4c33-a5e4-1959410d4071 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:30.662994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50818c0-cfff-4c33-a5e4-1959410d4071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:30.663757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50818c0-cfff-4c33-a5e4-1959410d4071 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=f9e5c137ffca43c1bbba55cc943e7814, user.id=b473d853ce2b4b828680acac501cc02d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:30.664064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50818c0-cfff-4c33-a5e4-1959410d4071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:30.665496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50818c0-cfff-4c33-a5e4-1959410d4071 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:30.670745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 589/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:30 2022] GET /identity/v3/role_assignments?user.id=b473d853ce2b4b828680acac501cc02d&scope.project.id=f9e5c137ffca43c1bbba55cc943e7814&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:32.613581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d57c9f-e7a2-4db5-beb2-8a60d2c47fac None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:32.614101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d57c9f-e7a2-4db5-beb2-8a60d2c47fac None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:32.614101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d57c9f-e7a2-4db5-beb2-8a60d2c47fac None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:32.615460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 558/1147] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:32.620490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264cca-d65b-455e-992c-ee06b794caed None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:32.620897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264cca-d65b-455e-992c-ee06b794caed None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:32.621049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264cca-d65b-455e-992c-ee06b794caed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:32.638864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-53264cca-d65b-455e-992c-ee06b794caed None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:32.640551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-53264cca-d65b-455e-992c-ee06b794caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:32.689266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:32.706614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 590/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:32.728876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d57c9f-e7a2-4db5-beb2-8a60d2c47fac None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:32.831692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d57c9f-e7a2-4db5-beb2-8a60d2c47fac None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:32.869879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaec7e-627d-4661-80ea-6bcc46bfe65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:32.870516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaec7e-627d-4661-80ea-6bcc46bfe65e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:32.870800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaec7e-627d-4661-80ea-6bcc46bfe65e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:32.871079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaec7e-627d-4661-80ea-6bcc46bfe65e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:32.872916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:32.882496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaec7e-627d-4661-80ea-6bcc46bfe65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-61aaec7e-627d-4661-80ea-6bcc46bfe65e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:32.887393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:32.888517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:32.890308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 559/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:32.895291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53264cca-d65b-455e-992c-ee06b794caed None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:32.990262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53264cca-d65b-455e-992c-ee06b794caed None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:33.023655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:33.024262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:33.024545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:33.024816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:33.025547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:33.025849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:33.027189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:33.035173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 591/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:33.041081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaec7e-627d-4661-80ea-6bcc46bfe65e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:33.137861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaec7e-627d-4661-80ea-6bcc46bfe65e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:33.171534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b61bc-f7f4-491b-9fe7-b7225500c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:33.172154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b61bc-f7f4-491b-9fe7-b7225500c675 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:33.172430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b61bc-f7f4-491b-9fe7-b7225500c675 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:33.172700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b61bc-f7f4-491b-9fe7-b7225500c675 None admin] PATH_INFO: `/v3/users/b473d853ce2b4b828680acac501cc02d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:33.173865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b61bc-f7f4-491b-9fe7-b7225500c675 None admin] RBAC: Authorizing `identity:get_user(user_id=b473d853ce2b4b828680acac501cc02d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:33.175289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b61bc-f7f4-491b-9fe7-b7225500c675 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:33.176866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 560/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:33 2022] GET /identity/v3/users/b473d853ce2b4b828680acac501cc02d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:33.182939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:33.231340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:33.284737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6216d-79f9-4411-8fde-144d1b5e5c30 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:33.318173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c0c0b-5697-46a0-afb0-bda59a476239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:33.318804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c0c0b-5697-46a0-afb0-bda59a476239 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:33.319083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c0c0b-5697-46a0-afb0-bda59a476239 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:33.319354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c0c0b-5697-46a0-afb0-bda59a476239 None admin] PATH_INFO: `/v3/projects/f9e5c137ffca43c1bbba55cc943e7814` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:33.320487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c0c0b-5697-46a0-afb0-bda59a476239 None admin] RBAC: Authorizing `identity:get_project(project_id=f9e5c137ffca43c1bbba55cc943e7814)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:33.321935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c0c0b-5697-46a0-afb0-bda59a476239 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:33.323846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 592/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:33 2022] GET /identity/v3/projects/f9e5c137ffca43c1bbba55cc943e7814 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:33.330006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b61bc-f7f4-491b-9fe7-b7225500c675 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:33.389709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:33.428153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b61bc-f7f4-491b-9fe7-b7225500c675 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:33.463095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e9dd2-bcaa-431e-9e7d-91faabbf795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:33.463754 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9dd2-bcaa-431e-9e7d-91faabbf795d None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:33.464033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9dd2-bcaa-431e-9e7d-91faabbf795d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:33.464305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9dd2-bcaa-431e-9e7d-91faabbf795d None admin] PATH_INFO: `/v3/projects/f9e5c137ffca43c1bbba55cc943e7814/users/b473d853ce2b4b828680acac501cc02d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:33.466258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e9dd2-bcaa-431e-9e7d-91faabbf795d None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e5c137ffca43c1bbba55cc943e7814, user_id=b473d853ce2b4b828680acac501cc02d, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:33.467686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e9dd2-bcaa-431e-9e7d-91faabbf795d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:33.480469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 561/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:25:33 2022] PUT /identity/v3/projects/f9e5c137ffca43c1bbba55cc943e7814/users/b473d853ce2b4b828680acac501cc02d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:25:35.467445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd8b60-75e6-4d0b-a7b4-e86d590b2a2d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:35.467912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd8b60-75e6-4d0b-a7b4-e86d590b2a2d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:35.468006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd8b60-75e6-4d0b-a7b4-e86d590b2a2d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:35.469357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 593/1154] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:35.474652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be286ce-aaaf-4272-9d63-d476be7accab None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:35.475059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be286ce-aaaf-4272-9d63-d476be7accab None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:35.475212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be286ce-aaaf-4272-9d63-d476be7accab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:35.493687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5be286ce-aaaf-4272-9d63-d476be7accab None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:35.494886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5be286ce-aaaf-4272-9d63-d476be7accab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:35.569183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 562/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:35.593605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd8b60-75e6-4d0b-a7b4-e86d590b2a2d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:35.695766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd8b60-75e6-4d0b-a7b4-e86d590b2a2d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:35.712188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:35.729993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40255b-1ac8-4dc4-9db7-816da61ae0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:35.730629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40255b-1ac8-4dc4-9db7-816da61ae0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:35.730909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40255b-1ac8-4dc4-9db7-816da61ae0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:35.731182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40255b-1ac8-4dc4-9db7-816da61ae0f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:35.741817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40255b-1ac8-4dc4-9db7-816da61ae0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0f40255b-1ac8-4dc4-9db7-816da61ae0f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:35.746958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:35.748154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:35.749627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 594/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:35.754710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be286ce-aaaf-4272-9d63-d476be7accab None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:35.809257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:35.856132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be286ce-aaaf-4272-9d63-d476be7accab None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:35.890396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:35.891015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:35.891299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:35.891574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:35.892286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:35.892592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:35.893979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:35.901895 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 563/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:35.908447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40255b-1ac8-4dc4-9db7-816da61ae0f3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:36.005355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40255b-1ac8-4dc4-9db7-816da61ae0f3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:36.037959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42593-d54b-4b82-b6ce-4f580ce8b04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:36.038586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42593-d54b-4b82-b6ce-4f580ce8b04b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:36.038870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42593-d54b-4b82-b6ce-4f580ce8b04b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:36.039146 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42593-d54b-4b82-b6ce-4f580ce8b04b None admin] PATH_INFO: `/v3/users/b473d853ce2b4b828680acac501cc02d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:36.040302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf42593-d54b-4b82-b6ce-4f580ce8b04b None admin] RBAC: Authorizing `identity:get_user(user_id=b473d853ce2b4b828680acac501cc02d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:36.041752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf42593-d54b-4b82-b6ce-4f580ce8b04b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:36.043329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 595/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:35 2022] GET /identity/v3/users/b473d853ce2b4b828680acac501cc02d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:36.049582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:36.147605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cecb87-8c7d-4b30-9941-b324b7ae40f5 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:36.162838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:36.182967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45b30c-a879-4d07-ac9c-180c320385f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:36.183588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45b30c-a879-4d07-ac9c-180c320385f9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:36.183871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45b30c-a879-4d07-ac9c-180c320385f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:36.184145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45b30c-a879-4d07-ac9c-180c320385f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:36.250221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45b30c-a879-4d07-ac9c-180c320385f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:36.251663 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45b30c-a879-4d07-ac9c-180c320385f9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:36.353957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 564/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:36.359900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42593-d54b-4b82-b6ce-4f580ce8b04b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:36.403751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:36.458064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42593-d54b-4b82-b6ce-4f580ce8b04b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:36.491884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6c4c1-5131-4309-baa6-6111538df333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:36.492542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6c4c1-5131-4309-baa6-6111538df333 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:36.492788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6c4c1-5131-4309-baa6-6111538df333 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:36.493060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6c4c1-5131-4309-baa6-6111538df333 None admin] PATH_INFO: `/v3/projects/f9e5c137ffca43c1bbba55cc943e7814` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:36.494225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6c4c1-5131-4309-baa6-6111538df333 None admin] RBAC: Authorizing `identity:get_project(project_id=f9e5c137ffca43c1bbba55cc943e7814)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:36.495642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6c4c1-5131-4309-baa6-6111538df333 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:36.497287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 596/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:36 2022] GET /identity/v3/projects/f9e5c137ffca43c1bbba55cc943e7814 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:36.504979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45b30c-a879-4d07-ac9c-180c320385f9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:36.520451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:36.604149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45b30c-a879-4d07-ac9c-180c320385f9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:36.636938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f741f-9c20-47ad-bb2d-ea0ed9e2e15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:36.637580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f741f-9c20-47ad-bb2d-ea0ed9e2e15a None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:36.637864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f741f-9c20-47ad-bb2d-ea0ed9e2e15a None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:36.638141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f741f-9c20-47ad-bb2d-ea0ed9e2e15a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:36.639761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f741f-9c20-47ad-bb2d-ea0ed9e2e15a None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=f9e5c137ffca43c1bbba55cc943e7814, user.id=b473d853ce2b4b828680acac501cc02d) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:36.640087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f741f-9c20-47ad-bb2d-ea0ed9e2e15a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:36.641541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f741f-9c20-47ad-bb2d-ea0ed9e2e15a None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:36.647151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 565/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:36 2022] GET /identity/v3/role_assignments?user.id=b473d853ce2b4b828680acac501cc02d&scope.project.id=f9e5c137ffca43c1bbba55cc943e7814&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:38.635831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7400b9-5a95-45bf-8f6f-ba04b6c82d6f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:38.636349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7400b9-5a95-45bf-8f6f-ba04b6c82d6f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:38.636349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7400b9-5a95-45bf-8f6f-ba04b6c82d6f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:38.637739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 597/1162] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:38.643496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c5f06-5167-40c1-8719-1cfef7ba77e2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:38.643880 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c5f06-5167-40c1-8719-1cfef7ba77e2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:38.644045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c5f06-5167-40c1-8719-1cfef7ba77e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:38.662969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c69c5f06-5167-40c1-8719-1cfef7ba77e2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:38.664132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c69c5f06-5167-40c1-8719-1cfef7ba77e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:38.727991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 566/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:38.749968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7400b9-5a95-45bf-8f6f-ba04b6c82d6f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:38.850754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7400b9-5a95-45bf-8f6f-ba04b6c82d6f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:38.863540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:38.884354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcac72-303f-40fd-b760-106c0aa94a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:38.884966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcac72-303f-40fd-b760-106c0aa94a6b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:38.885243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcac72-303f-40fd-b760-106c0aa94a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:38.885543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcac72-303f-40fd-b760-106c0aa94a6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:38.950571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcac72-303f-40fd-b760-106c0aa94a6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:38.952702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcac72-303f-40fd-b760-106c0aa94a6b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:39.047141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 598/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:39.053109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c5f06-5167-40c1-8719-1cfef7ba77e2 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:39.063691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:39.149543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c5f06-5167-40c1-8719-1cfef7ba77e2 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:39.181790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242aefa-8f17-4a8c-ad17-765039f881fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:39.182408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242aefa-8f17-4a8c-ad17-765039f881fd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:39.182684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242aefa-8f17-4a8c-ad17-765039f881fd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:39.182952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242aefa-8f17-4a8c-ad17-765039f881fd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:39.189307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242aefa-8f17-4a8c-ad17-765039f881fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:39.190751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242aefa-8f17-4a8c-ad17-765039f881fd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-7242aefa-8f17-4a8c-ad17-765039f881fd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:39.202592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:39.203916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:39.204820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 567/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:25:39 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:39.209839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcac72-303f-40fd-b760-106c0aa94a6b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:39.224304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:39.307344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcac72-303f-40fd-b760-106c0aa94a6b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:39.339929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b1153-4530-4d2e-8feb-c3ec8d229287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:39.340535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b1153-4530-4d2e-8feb-c3ec8d229287 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:39.340814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b1153-4530-4d2e-8feb-c3ec8d229287 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:39.341086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b1153-4530-4d2e-8feb-c3ec8d229287 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:39.341792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b1153-4530-4d2e-8feb-c3ec8d229287 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:39.342679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b1153-4530-4d2e-8feb-c3ec8d229287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:39.344019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b1153-4530-4d2e-8feb-c3ec8d229287 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:39.350897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 599/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Jul 28 06:25:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:41.334955 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cc046-122d-4625-91ac-c077b26e4f8d None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:41.335484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cc046-122d-4625-91ac-c077b26e4f8d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:41.335484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cc046-122d-4625-91ac-c077b26e4f8d None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:41.336843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 568/1167] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:41.342525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79ab76-37e0-428f-b133-b9aebd1cbd20 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:41.342958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79ab76-37e0-428f-b133-b9aebd1cbd20 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:41.343150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79ab76-37e0-428f-b133-b9aebd1cbd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:41.361075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bb79ab76-37e0-428f-b133-b9aebd1cbd20 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:41.362277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bb79ab76-37e0-428f-b133-b9aebd1cbd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:41.425473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 600/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:41.448351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cc046-122d-4625-91ac-c077b26e4f8d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:41.515289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:41.548602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cc046-122d-4625-91ac-c077b26e4f8d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:41.581538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326de66-164e-45ba-a3c1-67e8ac7b3f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:41.582143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326de66-164e-45ba-a3c1-67e8ac7b3f6b None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:41.582426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326de66-164e-45ba-a3c1-67e8ac7b3f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:41.582701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326de66-164e-45ba-a3c1-67e8ac7b3f6b None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:41.583404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326de66-164e-45ba-a3c1-67e8ac7b3f6b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:41.584758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326de66-164e-45ba-a3c1-67e8ac7b3f6b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:41.600734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 569/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Thu Jul 28 06:25:41 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:25:43.674956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ecfd-bb6b-4d0c-b4ee-9f81aeb0851f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:43.675478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ecfd-bb6b-4d0c-b4ee-9f81aeb0851f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:43.675566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ecfd-bb6b-4d0c-b4ee-9f81aeb0851f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:43.676859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 601/1170] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:43.682153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f91d-387f-4b8a-80cb-9709a4bc6931 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:43.682522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f91d-387f-4b8a-80cb-9709a4bc6931 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:43.682732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f91d-387f-4b8a-80cb-9709a4bc6931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:43.701185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d1d9f91d-387f-4b8a-80cb-9709a4bc6931 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:43.702404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d1d9f91d-387f-4b8a-80cb-9709a4bc6931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:43.766883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 570/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:43.789881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0ecfd-bb6b-4d0c-b4ee-9f81aeb0851f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:43.799661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:43.893714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0ecfd-bb6b-4d0c-b4ee-9f81aeb0851f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:43.926690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac96c7e-4ce5-4429-87e9-3d0da8e444c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:43.927324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96c7e-4ce5-4429-87e9-3d0da8e444c1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:43.927610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96c7e-4ce5-4429-87e9-3d0da8e444c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:43.927889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96c7e-4ce5-4429-87e9-3d0da8e444c1 None admin] PATH_INFO: `/v3/domains/d5c5ca5e12fd4706bf0735b373489f9e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:43.938686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac96c7e-4ce5-4429-87e9-3d0da8e444c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d5c5ca5e12fd4706bf0735b373489f9e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:43.940117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac96c7e-4ce5-4429-87e9-3d0da8e444c1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:43.941746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 602/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Thu Jul 28 06:25:43 2022] GET /identity/v3/domains/d5c5ca5e12fd4706bf0735b373489f9e => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:43.947544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9f91d-387f-4b8a-80cb-9709a4bc6931 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:44.004044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:44.044590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9f91d-387f-4b8a-80cb-9709a4bc6931 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:44.077661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a4c5b-58f0-46b5-b765-23603e5d8bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:44.078268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a4c5b-58f0-46b5-b765-23603e5d8bca None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:44.078547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a4c5b-58f0-46b5-b765-23603e5d8bca None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:44.078817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a4c5b-58f0-46b5-b765-23603e5d8bca None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:44.079485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a4c5b-58f0-46b5-b765-23603e5d8bca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:44.080891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a4c5b-58f0-46b5-b765-23603e5d8bca None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:44.105968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 571/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:25:43 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:25:46.115806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56450aca-5f53-489f-8874-54d3b5664304 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:46.116392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56450aca-5f53-489f-8874-54d3b5664304 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:46.116392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56450aca-5f53-489f-8874-54d3b5664304 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:46.117734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 603/1174] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:46.123051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecec54-2445-4928-86bf-e723a5efbcaf None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:46.123528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecec54-2445-4928-86bf-e723a5efbcaf None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:46.123835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecec54-2445-4928-86bf-e723a5efbcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:46.142555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-18ecec54-2445-4928-86bf-e723a5efbcaf None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:46.143736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-18ecec54-2445-4928-86bf-e723a5efbcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:46.210387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 572/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:46.235422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56450aca-5f53-489f-8874-54d3b5664304 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:46.301446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:46.338257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56450aca-5f53-489f-8874-54d3b5664304 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:46.371453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b18f3-39da-43e2-98d9-1fe12765c2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:46.372016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b18f3-39da-43e2-98d9-1fe12765c2df None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:46.372293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b18f3-39da-43e2-98d9-1fe12765c2df None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:46.372564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b18f3-39da-43e2-98d9-1fe12765c2df None admin] PATH_INFO: `/v3/domains/d5c5ca5e12fd4706bf0735b373489f9e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:46.374527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b18f3-39da-43e2-98d9-1fe12765c2df None admin] RBAC: Authorizing `identity:get_domain(domain_id=d5c5ca5e12fd4706bf0735b373489f9e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:46.376232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b18f3-39da-43e2-98d9-1fe12765c2df None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:46.377835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 604/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Thu Jul 28 06:25:46 2022] GET /identity/v3/domains/d5c5ca5e12fd4706bf0735b373489f9e => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:46.383660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecec54-2445-4928-86bf-e723a5efbcaf None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:46.425052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:46.483492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecec54-2445-4928-86bf-e723a5efbcaf None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:46.517088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b0073-3aec-4174-8e0d-422a79f8960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:46.517728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b0073-3aec-4174-8e0d-422a79f8960d None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:46.518015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b0073-3aec-4174-8e0d-422a79f8960d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:46.518289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b0073-3aec-4174-8e0d-422a79f8960d None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:46.518977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b0073-3aec-4174-8e0d-422a79f8960d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:46.520397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b0073-3aec-4174-8e0d-422a79f8960d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:46.542106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 573/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:25:46 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:25:48.566962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386b8d4-48d4-4553-9a29-cf009ac79c16 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:48.567510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386b8d4-48d4-4553-9a29-cf009ac79c16 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:48.567510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386b8d4-48d4-4553-9a29-cf009ac79c16 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:48.568839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 605/1178] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:48.574121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c2d0d-a0e5-45d7-870e-3fbe4c882d13 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:48.574558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c2d0d-a0e5-45d7-870e-3fbe4c882d13 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:48.574668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c2d0d-a0e5-45d7-870e-3fbe4c882d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:48.593020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d36c2d0d-a0e5-45d7-870e-3fbe4c882d13 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:48.594224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d36c2d0d-a0e5-45d7-870e-3fbe4c882d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:48.659174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 574/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:48.682314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386b8d4-48d4-4553-9a29-cf009ac79c16 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:48.778516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386b8d4-48d4-4553-9a29-cf009ac79c16 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:48.811425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afa35f-027b-4c75-9358-db0268f33ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:48.812044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa35f-027b-4c75-9358-db0268f33ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:48.812319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa35f-027b-4c75-9358-db0268f33ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:48.812584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa35f-027b-4c75-9358-db0268f33ef6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:48.814404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:48.823331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afa35f-027b-4c75-9358-db0268f33ef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-58afa35f-027b-4c75-9358-db0268f33ef6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:48.828717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:48.829984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:48.831128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 606/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:48.835951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c2d0d-a0e5-45d7-870e-3fbe4c882d13 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:48.862523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:48.937543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c2d0d-a0e5-45d7-870e-3fbe4c882d13 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:48.971178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:48.971789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:48.972073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:48.972347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:48.973054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:48.973394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:48.974758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:48.983301 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 575/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:48.989070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afa35f-027b-4c75-9358-db0268f33ef6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:49.086383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afa35f-027b-4c75-9358-db0268f33ef6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:49.119937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4e27-9f38-4cb0-aae6-22704c3853d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:49.120561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4e27-9f38-4cb0-aae6-22704c3853d2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:49.120843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4e27-9f38-4cb0-aae6-22704c3853d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:49.121124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4e27-9f38-4cb0-aae6-22704c3853d2 None admin] PATH_INFO: `/v3/users/73ddfa6a1bdd4026952b90932e836fc3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:49.134564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd4e27-9f38-4cb0-aae6-22704c3853d2 None admin] RBAC: Authorizing `identity:get_user(user_id=73ddfa6a1bdd4026952b90932e836fc3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:49.136005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd4e27-9f38-4cb0-aae6-22704c3853d2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:49.137647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 607/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:48 2022] GET /identity/v3/users/73ddfa6a1bdd4026952b90932e836fc3 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:49.143669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:49.227315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:49.239637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e8688-18b1-41f7-be28-66378422ee0e None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:49.272931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f9e21-c0b7-41b4-a91c-11f2a00d2913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:49.273567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f9e21-c0b7-41b4-a91c-11f2a00d2913 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:49.273847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f9e21-c0b7-41b4-a91c-11f2a00d2913 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:49.274120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f9e21-c0b7-41b4-a91c-11f2a00d2913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:49.339329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f9e21-c0b7-41b4-a91c-11f2a00d2913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:49.340710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f9e21-c0b7-41b4-a91c-11f2a00d2913 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:49.423169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:49.438090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 576/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:49.444039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4e27-9f38-4cb0-aae6-22704c3853d2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:49.495551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:49.542227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4e27-9f38-4cb0-aae6-22704c3853d2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:49.574744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50266ae-1a1f-49d6-8526-76d806bf8822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:49.575382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50266ae-1a1f-49d6-8526-76d806bf8822 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:49.575661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50266ae-1a1f-49d6-8526-76d806bf8822 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:49.576624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50266ae-1a1f-49d6-8526-76d806bf8822 None admin] PATH_INFO: `/v3/projects/a4b1624ed06248d6bab4fa1aafbc3f64` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:49.577879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50266ae-1a1f-49d6-8526-76d806bf8822 None admin] RBAC: Authorizing `identity:get_project(project_id=a4b1624ed06248d6bab4fa1aafbc3f64)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:49.579291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50266ae-1a1f-49d6-8526-76d806bf8822 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:49.580914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 608/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:49 2022] GET /identity/v3/projects/a4b1624ed06248d6bab4fa1aafbc3f64 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:49.587038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f9e21-c0b7-41b4-a91c-11f2a00d2913 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:49.684433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f9e21-c0b7-41b4-a91c-11f2a00d2913 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:49.720183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378b310-1e9d-48d9-a598-f595bdb80319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:49.720807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378b310-1e9d-48d9-a598-f595bdb80319 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:49.721091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378b310-1e9d-48d9-a598-f595bdb80319 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:49.721393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378b310-1e9d-48d9-a598-f595bdb80319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:49.722189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378b310-1e9d-48d9-a598-f595bdb80319 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=a4b1624ed06248d6bab4fa1aafbc3f64, user.id=73ddfa6a1bdd4026952b90932e836fc3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:49.722508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378b310-1e9d-48d9-a598-f595bdb80319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:49.723947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378b310-1e9d-48d9-a598-f595bdb80319 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:49.729444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 577/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:49 2022] GET /identity/v3/role_assignments?user.id=73ddfa6a1bdd4026952b90932e836fc3&scope.project.id=a4b1624ed06248d6bab4fa1aafbc3f64&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:51.786496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d623-6c61-47ce-9067-f0f8ecb4f502 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:51.787077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d623-6c61-47ce-9067-f0f8ecb4f502 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:51.787077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d623-6c61-47ce-9067-f0f8ecb4f502 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:51.788377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 609/1186] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:51.793784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a41f-59d1-458a-b883-e7fb757e615d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:51.794188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a41f-59d1-458a-b883-e7fb757e615d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:51.794348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a41f-59d1-458a-b883-e7fb757e615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:51.812398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-aa20a41f-59d1-458a-b883-e7fb757e615d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:51.813617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-aa20a41f-59d1-458a-b883-e7fb757e615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:51.859854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:51.880152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 578/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:51.902703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8d623-6c61-47ce-9067-f0f8ecb4f502 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:52.009473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8d623-6c61-47ce-9067-f0f8ecb4f502 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:52.029067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:52.043182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb81a2-172f-42e5-ac1b-fed4efe1ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:52.043753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb81a2-172f-42e5-ac1b-fed4efe1ae14 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:52.044033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb81a2-172f-42e5-ac1b-fed4efe1ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:52.044304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb81a2-172f-42e5-ac1b-fed4efe1ae14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:52.055014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb81a2-172f-42e5-ac1b-fed4efe1ae14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-dddb81a2-172f-42e5-ac1b-fed4efe1ae14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:52.059858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:52.061659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:25:52.062619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 610/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:52.068235 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a41f-59d1-458a-b883-e7fb757e615d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:52.165047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a41f-59d1-458a-b883-e7fb757e615d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:52.198859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:52.199464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:52.199749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:52.200026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:52.200733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:52.201061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:52.202454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:52.210210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 579/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:52.215885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb81a2-172f-42e5-ac1b-fed4efe1ae14 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:52.319988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb81a2-172f-42e5-ac1b-fed4efe1ae14 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:52.353508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee637e67-7a02-4979-b72e-6a9e4527f3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:52.354387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee637e67-7a02-4979-b72e-6a9e4527f3ff None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:52.354514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee637e67-7a02-4979-b72e-6a9e4527f3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:52.354716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee637e67-7a02-4979-b72e-6a9e4527f3ff None admin] PATH_INFO: `/v3/users/73ddfa6a1bdd4026952b90932e836fc3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:52.355874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee637e67-7a02-4979-b72e-6a9e4527f3ff None admin] RBAC: Authorizing `identity:get_user(user_id=73ddfa6a1bdd4026952b90932e836fc3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:52.357961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee637e67-7a02-4979-b72e-6a9e4527f3ff None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:52.359722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 611/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:52 2022] GET /identity/v3/users/73ddfa6a1bdd4026952b90932e836fc3 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:52.365604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:52.410853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:52.465495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd0609-3cf5-4971-a35f-fd2feae416cc None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:52.498815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9843cce-73a1-44f3-bd2c-f021803a143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:52.499448 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843cce-73a1-44f3-bd2c-f021803a143d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:52.499729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843cce-73a1-44f3-bd2c-f021803a143d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:52.500003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843cce-73a1-44f3-bd2c-f021803a143d None admin] PATH_INFO: `/v3/projects/a4b1624ed06248d6bab4fa1aafbc3f64` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:52.501137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9843cce-73a1-44f3-bd2c-f021803a143d None admin] RBAC: Authorizing `identity:get_project(project_id=a4b1624ed06248d6bab4fa1aafbc3f64)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:52.502585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9843cce-73a1-44f3-bd2c-f021803a143d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:52.504244 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 580/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:52 2022] GET /identity/v3/projects/a4b1624ed06248d6bab4fa1aafbc3f64 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:52.510247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee637e67-7a02-4979-b72e-6a9e4527f3ff None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:52.558242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:52.623292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee637e67-7a02-4979-b72e-6a9e4527f3ff None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:52.656886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce429ed-d426-4f6f-b538-86e1a7bc6f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:52.657573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce429ed-d426-4f6f-b538-86e1a7bc6f21 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:52.657855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce429ed-d426-4f6f-b538-86e1a7bc6f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:52.658128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce429ed-d426-4f6f-b538-86e1a7bc6f21 None admin] PATH_INFO: `/v3/projects/a4b1624ed06248d6bab4fa1aafbc3f64/users/73ddfa6a1bdd4026952b90932e836fc3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:52.660041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce429ed-d426-4f6f-b538-86e1a7bc6f21 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b1624ed06248d6bab4fa1aafbc3f64, user_id=73ddfa6a1bdd4026952b90932e836fc3, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:52.661494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce429ed-d426-4f6f-b538-86e1a7bc6f21 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:52.673792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 612/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:25:52 2022] PUT /identity/v3/projects/a4b1624ed06248d6bab4fa1aafbc3f64/users/73ddfa6a1bdd4026952b90932e836fc3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:25:54.649393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5b8f-f091-41b3-b8cc-181458e86c94 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:54.649912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5b8f-f091-41b3-b8cc-181458e86c94 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:54.650000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5b8f-f091-41b3-b8cc-181458e86c94 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:54.651342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 581/1193] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:25:54.656979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6924-7015-4dc2-863d-ad1829887f6d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:54.657425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6924-7015-4dc2-863d-ad1829887f6d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:54.657572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6924-7015-4dc2-863d-ad1829887f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:54.675231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a56d6924-7015-4dc2-863d-ad1829887f6d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:25:54.676411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a56d6924-7015-4dc2-863d-ad1829887f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:25:54.740806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 613/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:25:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:25:54.765161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b5b8f-f091-41b3-b8cc-181458e86c94 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:54.864315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b5b8f-f091-41b3-b8cc-181458e86c94 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:54.877382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:54.898647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29307400-0ddd-4a00-b043-0a4b10f42ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:54.899282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29307400-0ddd-4a00-b043-0a4b10f42ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:54.899565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29307400-0ddd-4a00-b043-0a4b10f42ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:54.899841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29307400-0ddd-4a00-b043-0a4b10f42ea1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:54.909750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29307400-0ddd-4a00-b043-0a4b10f42ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-29307400-0ddd-4a00-b043-0a4b10f42ea1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:25:54.914646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:25:54.915752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:25:54.916706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 582/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:25:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:25:54.922597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6924-7015-4dc2-863d-ad1829887f6d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:54.962077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:55.020099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6924-7015-4dc2-863d-ad1829887f6d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:55.053082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:55.053727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:55.054010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:55.054284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:55.054991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:55.055296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:55.056656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:55.065745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 614/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:25:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:55.071489 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29307400-0ddd-4a00-b043-0a4b10f42ea1 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:55.170851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29307400-0ddd-4a00-b043-0a4b10f42ea1 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:55.204080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8fd9b-488a-4423-9b0d-83ad20d7a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:55.204709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8fd9b-488a-4423-9b0d-83ad20d7a039 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:55.204987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8fd9b-488a-4423-9b0d-83ad20d7a039 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:55.205260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8fd9b-488a-4423-9b0d-83ad20d7a039 None admin] PATH_INFO: `/v3/users/73ddfa6a1bdd4026952b90932e836fc3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:55.206512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8fd9b-488a-4423-9b0d-83ad20d7a039 None admin] RBAC: Authorizing `identity:get_user(user_id=73ddfa6a1bdd4026952b90932e836fc3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:55.207952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8fd9b-488a-4423-9b0d-83ad20d7a039 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:55.209600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 583/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Jul 28 06:25:55 2022] GET /identity/v3/users/73ddfa6a1bdd4026952b90932e836fc3 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:55.215355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:55.310286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:55.313143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740ff17-21c6-492d-8fd8-0a976a0b5049 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:55.347346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822cb77-0e00-4fdf-b36e-22f8aaf2c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:55.347944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822cb77-0e00-4fdf-b36e-22f8aaf2c64c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:55.348225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822cb77-0e00-4fdf-b36e-22f8aaf2c64c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:55.348499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822cb77-0e00-4fdf-b36e-22f8aaf2c64c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:55.415050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822cb77-0e00-4fdf-b36e-22f8aaf2c64c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:55.416486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822cb77-0e00-4fdf-b36e-22f8aaf2c64c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:55.512682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:55.517249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 615/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:25:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:25:55.522682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8fd9b-488a-4423-9b0d-83ad20d7a039 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:55.564111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:25:55.619736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8fd9b-488a-4423-9b0d-83ad20d7a039 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:55.652358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd91ca-1f55-4319-ba54-f5df1f0c7b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:55.652972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd91ca-1f55-4319-ba54-f5df1f0c7b47 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:55.653246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd91ca-1f55-4319-ba54-f5df1f0c7b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:55.653542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd91ca-1f55-4319-ba54-f5df1f0c7b47 None admin] PATH_INFO: `/v3/projects/a4b1624ed06248d6bab4fa1aafbc3f64` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:55.655546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd91ca-1f55-4319-ba54-f5df1f0c7b47 None admin] RBAC: Authorizing `identity:get_project(project_id=a4b1624ed06248d6bab4fa1aafbc3f64)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:55.656966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd91ca-1f55-4319-ba54-f5df1f0c7b47 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:55.658622 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 584/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Jul 28 06:25:55 2022] GET /identity/v3/projects/a4b1624ed06248d6bab4fa1aafbc3f64 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:25:55.664879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822cb77-0e00-4fdf-b36e-22f8aaf2c64c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:25:55.762097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822cb77-0e00-4fdf-b36e-22f8aaf2c64c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:25:55.796875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b5ad3-988a-4cd9-9aa9-233ae6e6a6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:25:55.797508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b5ad3-988a-4cd9-9aa9-233ae6e6a6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:25:55.797791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b5ad3-988a-4cd9-9aa9-233ae6e6a6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:25:55.798062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b5ad3-988a-4cd9-9aa9-233ae6e6a6a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:25:55.798839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b5ad3-988a-4cd9-9aa9-233ae6e6a6a1 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=a4b1624ed06248d6bab4fa1aafbc3f64, user.id=73ddfa6a1bdd4026952b90932e836fc3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:25:55.799146 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b5ad3-988a-4cd9-9aa9-233ae6e6a6a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:25:55.800552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b5ad3-988a-4cd9-9aa9-233ae6e6a6a1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:25:55.806199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 616/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:25:55 2022] GET /identity/v3/role_assignments?user.id=73ddfa6a1bdd4026952b90932e836fc3&scope.project.id=a4b1624ed06248d6bab4fa1aafbc3f64&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:05.648342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=46 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:05.803633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=68 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:51.247489 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf5201-139a-4ca6-a2f5-c5f8a5d940ac None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:51.248118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf5201-139a-4ca6-a2f5-c5f8a5d940ac None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:51.248118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf5201-139a-4ca6-a2f5-c5f8a5d940ac None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:51.249800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 585/1201] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:26:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:51.256750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664e6d-f85b-4f66-a37a-feb0b32d1139 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:51.257237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664e6d-f85b-4f66-a37a-feb0b32d1139 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:51.257439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664e6d-f85b-4f66-a37a-feb0b32d1139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:51.283290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4e664e6d-f85b-4f66-a37a-feb0b32d1139 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:51.284592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4e664e6d-f85b-4f66-a37a-feb0b32d1139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:51.370122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 617/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:26:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:26:51.397141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf5201-139a-4ca6-a2f5-c5f8a5d940ac None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:51.538709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf5201-139a-4ca6-a2f5-c5f8a5d940ac None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:51.583158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:51.583158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:51.583158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:51.583563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:51.597447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:51.599203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:26:51.615840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:26:51.616984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:26:51.618043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:26:51.618043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:26:51.618043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:26:51.618043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:26:51.627087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e664e6d-f85b-4f66-a37a-feb0b32d1139 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:51.630465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 586/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:26:51.764906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e664e6d-f85b-4f66-a37a-feb0b32d1139 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:51.804773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:51.813147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6c237-17bc-45f4-a728-14b70d8777cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:51.813925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6c237-17bc-45f4-a728-14b70d8777cf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:51.814305 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6c237-17bc-45f4-a728-14b70d8777cf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:51.814602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6c237-17bc-45f4-a728-14b70d8777cf None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:51.815563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6c237-17bc-45f4-a728-14b70d8777cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:26:51.815874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6c237-17bc-45f4-a728-14b70d8777cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:51.817245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6c237-17bc-45f4-a728-14b70d8777cf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:51.833039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 618/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:51.841712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:51.912967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:51.976120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7d5f8-244e-4e62-8b65-a39db93e820d None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:52.022015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fc783-f8ec-43e2-8d20-0f5ca308a1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:52.022610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fc783-f8ec-43e2-8d20-0f5ca308a1f3 None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:52.022981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fc783-f8ec-43e2-8d20-0f5ca308a1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:52.023183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fc783-f8ec-43e2-8d20-0f5ca308a1f3 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:52.023889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fc783-f8ec-43e2-8d20-0f5ca308a1f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:52.025341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fc783-f8ec-43e2-8d20-0f5ca308a1f3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:52.049943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 587/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Jul 28 06:26:51 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:26:54.395805 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06045945-30da-4a49-a416-b6f0b8ddb403 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:54.396368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06045945-30da-4a49-a416-b6f0b8ddb403 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:54.396368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06045945-30da-4a49-a416-b6f0b8ddb403 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:54.398181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 619/1206] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:54.404344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48f3b7-a6bd-4654-a89b-49517163de30 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:54.404798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48f3b7-a6bd-4654-a89b-49517163de30 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:54.404885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48f3b7-a6bd-4654-a89b-49517163de30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:54.426129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-db48f3b7-a6bd-4654-a89b-49517163de30 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:54.427313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-db48f3b7-a6bd-4654-a89b-49517163de30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:54.517732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 588/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:26:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:26:54.545132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06045945-30da-4a49-a416-b6f0b8ddb403 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:54.676377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06045945-30da-4a49-a416-b6f0b8ddb403 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:54.723808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291003-253c-4249-951e-a475f1d4d014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:54.724531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291003-253c-4249-951e-a475f1d4d014 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:54.724720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291003-253c-4249-951e-a475f1d4d014 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:54.724979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291003-253c-4249-951e-a475f1d4d014 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:54.736544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc291003-253c-4249-951e-a475f1d4d014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-dc291003-253c-4249-951e-a475f1d4d014 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:26:54.741550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:26:54.742540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:26:54.743419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 620/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:26:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:26:54.749989 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48f3b7-a6bd-4654-a89b-49517163de30 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:54.810089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:54.893100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48f3b7-a6bd-4654-a89b-49517163de30 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:54.936872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:54.936872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:54.936872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:54.936872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:54.939490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:26:54.939809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:54.941192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:54.951437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 589/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:26:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:54.958230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291003-253c-4249-951e-a475f1d4d014 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:55.023715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:55.086353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291003-253c-4249-951e-a475f1d4d014 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:55.127613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:55.128321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:55.128538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:55.128820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:55.141089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:55.142687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:26:55.158575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:26:55.159891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:26:55.160811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:26:55.160811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:26:55.160811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:26:55.160811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:26:55.160811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 621/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:26:55.166367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:55.285479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:55.289996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56b47-96f7-46d6-970b-c495301defe8 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:55.332068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:55.332703 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:55.332987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:55.333270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:55.335863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:26:55.335863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:55.335863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:55.348533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 590/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:55.355786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:55.473514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e123918-5d45-473a-b53a-1dfb0fb7b26e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:55.475104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:55.515086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:55.515737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:55.516059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:55.516396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:55.524829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:55.526343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:26:55.540790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:26:55.541881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:26:55.544026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 622/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:26:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:26:55.549093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:55.674125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a4a7c-3515-4020-be89-9d6872da78eb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:55.713860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:55.714592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:55.714875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:55.715168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:55.715943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:26:55.716254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:55.718009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:55.741372 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 591/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jul 28 06:26:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:55.748186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:55.879044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9a6b7-1366-40d9-acb1-ccc412a9aa0f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:55.921235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e616fcf-a15b-441f-8942-52521e39ee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:55.921971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e616fcf-a15b-441f-8942-52521e39ee96 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:55.922255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e616fcf-a15b-441f-8942-52521e39ee96 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:55.922531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e616fcf-a15b-441f-8942-52521e39ee96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:55.962752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:56.012960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e616fcf-a15b-441f-8942-52521e39ee96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:56.014869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e616fcf-a15b-441f-8942-52521e39ee96 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:56.152578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 623/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:26:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 06:26:56.158193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:56.183721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:56.277837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3342261-6fa1-4f08-be20-f556d038ffb6 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:56.319735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:56.320362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:56.320686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:56.320963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:56.333855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:56.335308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:26:56.351941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:26:56.353596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:26:56.354643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:26:56.354643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:26:56.354643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:26:56.354643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:26:56.354643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 592/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:26:56.359930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e616fcf-a15b-441f-8942-52521e39ee96 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:56.427238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:56.470057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e616fcf-a15b-441f-8942-52521e39ee96 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:56.508143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:56.508815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:56.509027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:56.509326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:56.510016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:26:56.510319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:56.512223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:56.525342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 624/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:56.529935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:56.608462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:56.646209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433d7a7-7506-4999-bdbe-d0b257796e3b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:56.691362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:56.692561 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:56.693217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:56.693778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:56.703132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:56.704771 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:26:56.718207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:26:56.719130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:26:56.725278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 593/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:26:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:26:56.726038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:56.837777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404b907-1a84-44d2-b697-d1ece9a0ee41 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:56.865915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:56.879786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381a535-26cc-42d0-bfed-2f082a9c73d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:56.880423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a535-26cc-42d0-bfed-2f082a9c73d3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:56.880746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a535-26cc-42d0-bfed-2f082a9c73d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:56.881097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a535-26cc-42d0-bfed-2f082a9c73d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:56.881938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381a535-26cc-42d0-bfed-2f082a9c73d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:26:56.882250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381a535-26cc-42d0-bfed-2f082a9c73d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:56.883888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381a535-26cc-42d0-bfed-2f082a9c73d3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:56.898488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 625/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:26:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:56.905125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:57.026185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f884eb-54f3-4c82-8a57-6b5eacd5b761 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:57.056092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:57.063870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd6d75-083e-4804-a890-35047d9de783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:57.064550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd6d75-083e-4804-a890-35047d9de783 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:57.064901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd6d75-083e-4804-a890-35047d9de783 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:57.065095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd6d75-083e-4804-a890-35047d9de783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:57.065959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd6d75-083e-4804-a890-35047d9de783 None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=07b18ac45ec948738129f3d08e0d0eb3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:26:57.066317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd6d75-083e-4804-a890-35047d9de783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:57.067830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd6d75-083e-4804-a890-35047d9de783 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:57.075045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 594/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:26:56 2022] GET /identity/v3/role_assignments?user.id=07b18ac45ec948738129f3d08e0d0eb3&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:59.363338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccb2d-5bf4-4247-a429-8ae890dfb513 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:59.364005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccb2d-5bf4-4247-a429-8ae890dfb513 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:59.364005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccb2d-5bf4-4247-a429-8ae890dfb513 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:59.365639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 626/1220] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:26:59.371829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c32fd-9da1-4813-9299-8d3c8cc4f84f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:59.372286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c32fd-9da1-4813-9299-8d3c8cc4f84f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:59.372402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c32fd-9da1-4813-9299-8d3c8cc4f84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:59.390754 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c45c32fd-9da1-4813-9299-8d3c8cc4f84f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:26:59.392545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c45c32fd-9da1-4813-9299-8d3c8cc4f84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:26:59.472112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 595/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:26:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:26:59.495221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbccb2d-5bf4-4247-a429-8ae890dfb513 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:59.610259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbccb2d-5bf4-4247-a429-8ae890dfb513 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:59.645848 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefb0c7-e1d5-415c-a3c7-2f6fcb6695ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:59.646407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefb0c7-e1d5-415c-a3c7-2f6fcb6695ad None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:59.646688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefb0c7-e1d5-415c-a3c7-2f6fcb6695ad None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:59.646971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefb0c7-e1d5-415c-a3c7-2f6fcb6695ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:59.657827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefb0c7-e1d5-415c-a3c7-2f6fcb6695ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ddefb0c7-e1d5-415c-a3c7-2f6fcb6695ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:26:59.662688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:26:59.663851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:26:59.665229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 627/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:26:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:26:59.670144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c32fd-9da1-4813-9299-8d3c8cc4f84f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:59.769454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c32fd-9da1-4813-9299-8d3c8cc4f84f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:59.794503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:59.805866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:59.806478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:59.806758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:59.807026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:59.807748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:26:59.808735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:59.810145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:26:59.823427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 596/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:26:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:26:59.824968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefb0c7-e1d5-415c-a3c7-2f6fcb6695ad None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:26:59.880221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:26:59.939862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefb0c7-e1d5-415c-a3c7-2f6fcb6695ad None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:26:59.976236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:26:59.976880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:26:59.977164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:26:59.977476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:26:59.987840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:26:59.989344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:00.006260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:00.007365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:00.013173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:00.013173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:00.013173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:00.013173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:00.013173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 628/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:00.014140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:00.124086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ff393-0273-4d38-a02d-398ff8a119b0 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:00.161046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:00.161737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:00.161985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:00.162294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:00.162954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:00.163263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:00.164642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:00.178426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 597/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:00.184987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:00.289507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:00.299037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919afc5-7008-4d52-ad07-b6626554e231 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:00.340962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:00.341634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:00.341913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:00.342184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:00.351863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:00.353414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:27:00.366646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:00.368483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:00.369822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 629/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:27:00.374852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:00.423942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:00.489263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5abfd-94ed-45f4-a740-3d7ed82b8695 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:00.531963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:00.532885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:00.533238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:00.533572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:00.534750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:00.535134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:00.536832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:00.557760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 598/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jul 28 06:27:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:00.565555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:00.683612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e028c-2420-47d7-9bb6-29e6146d7b99 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:00.721336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:00.721964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:00.722256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:00.722515 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:00.733077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:00.734555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:00.749233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:00.750491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:00.751719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:00.751719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:00.751719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:00.751719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:00.751719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 630/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:00.757070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:00.871223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7948be-ce3f-42ed-9244-bf9d8a89f063 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:00.874395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:00.913488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:00.914147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:00.914397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:00.914665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:00.915354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:00.915652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:00.917008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:00.929622 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 599/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:00.936204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:00.946803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:01.045831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b410bff-b615-4c9b-b1ab-657234335a06 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:01.083254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:01.083908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:01.084187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:01.084458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:01.092482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:01.093964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:27:01.106201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:01.107531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:01.108671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 631/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:27:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:01.114209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:01.238317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934cd84-af74-4da7-8f12-d5f3128e065b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:01.278696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cace4d-477e-4a83-94a1-d4f2c38e5176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:01.279634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cace4d-477e-4a83-94a1-d4f2c38e5176 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:01.280045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cace4d-477e-4a83-94a1-d4f2c38e5176 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:01.280438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cace4d-477e-4a83-94a1-d4f2c38e5176 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:01.281330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cace4d-477e-4a83-94a1-d4f2c38e5176 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:01.281646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cace4d-477e-4a83-94a1-d4f2c38e5176 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:01.283074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cace4d-477e-4a83-94a1-d4f2c38e5176 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:01.297090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 600/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:27:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:01.303378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:01.383541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:01.425792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccb941-3a68-453b-b6e8-a9b37c7cd9c7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:01.462541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d452377-7fd4-4831-8d69-5a19ff511ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:01.463273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452377-7fd4-4831-8d69-5a19ff511ceb None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:01.463579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452377-7fd4-4831-8d69-5a19ff511ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:01.463909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452377-7fd4-4831-8d69-5a19ff511ceb None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/07b18ac45ec948738129f3d08e0d0eb3/roles/e6fe0c4d4ed1454f935b5c9001151260` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:01.480131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d452377-7fd4-4831-8d69-5a19ff511ceb None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=07b18ac45ec948738129f3d08e0d0eb3, role_id=e6fe0c4d4ed1454f935b5c9001151260)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:01.481605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d452377-7fd4-4831-8d69-5a19ff511ceb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:01.497549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 632/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:27:01 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/07b18ac45ec948738129f3d08e0d0eb3/roles/e6fe0c4d4ed1454f935b5c9001151260 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:27:03.741502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e715785-0f08-4ec0-ba16-4c6547e14575 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:03.742061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e715785-0f08-4ec0-ba16-4c6547e14575 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:03.742061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e715785-0f08-4ec0-ba16-4c6547e14575 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:03.743624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 601/1233] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:27:03.749598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49b5e-2ba4-47ed-8df9-3eff0c27495f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:03.750042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49b5e-2ba4-47ed-8df9-3eff0c27495f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:03.750150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49b5e-2ba4-47ed-8df9-3eff0c27495f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:03.769108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-69a49b5e-2ba4-47ed-8df9-3eff0c27495f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:27:03.770341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-69a49b5e-2ba4-47ed-8df9-3eff0c27495f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:27:03.846122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 633/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:27:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:27:03.870933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e715785-0f08-4ec0-ba16-4c6547e14575 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:03.902688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:03.990191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e715785-0f08-4ec0-ba16-4c6547e14575 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:04.027907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26284b-849c-4f1e-a16f-1621634434c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:04.028512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26284b-849c-4f1e-a16f-1621634434c4 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:04.028874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26284b-849c-4f1e-a16f-1621634434c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:04.029154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26284b-849c-4f1e-a16f-1621634434c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:04.040641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26284b-849c-4f1e-a16f-1621634434c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-aa26284b-849c-4f1e-a16f-1621634434c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:27:04.046695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:04.048126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:04.049341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 602/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Jul 28 06:27:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:04.054684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49b5e-2ba4-47ed-8df9-3eff0c27495f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:04.105089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:04.166260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49b5e-2ba4-47ed-8df9-3eff0c27495f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:04.208052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:04.208820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:04.209221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:04.209627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:04.210609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:04.211026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:04.212606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:04.221755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 634/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Jul 28 06:27:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:27:04.228168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26284b-849c-4f1e-a16f-1621634434c4 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:04.330436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:04.354138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26284b-849c-4f1e-a16f-1621634434c4 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:04.392184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:04.392830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:04.393114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:04.393420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:04.406742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:04.408631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:04.425826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:04.426981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:04.428048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:04.428048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:04.428048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:04.428048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:04.428048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 603/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:04.434065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:04.559238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8ed9-e1b9-4790-825c-7645f24df80e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:04.569112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:04.602095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:04.602680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:04.603249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:04.603541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:04.605129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:04.606166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:04.608198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:04.621611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 635/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:27:04.627038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:04.750611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176e01-0f59-452f-81d6-11de790ea810 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:04.791640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:04.793018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:04.793373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:04.793651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:04.796851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:04.803313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:04.805092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:27:04.818748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:04.820497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:04.821786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 604/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:04.826735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:04.945232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94568d-4e56-47f4-ab7b-8c3528e27561 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:04.986630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:04.987576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:04.987856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:04.988130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:04.989136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:04.989487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:04.990968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:05.010495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 636/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jul 28 06:27:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:05.016889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:05.131063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d5c81-b349-4e9a-9cc4-e8ef54d92693 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:05.183685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b0645-d703-4264-8c68-0aca85efbf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:05.185407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b0645-d703-4264-8c68-0aca85efbf63 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:05.185871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b0645-d703-4264-8c68-0aca85efbf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:05.186308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b0645-d703-4264-8c68-0aca85efbf63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:05.264431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:05.271599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b0645-d703-4264-8c68-0aca85efbf63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:05.273046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b0645-d703-4264-8c68-0aca85efbf63 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:05.389985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 605/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:27:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:27:05.396247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:05.432968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:05.525050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a056d4-f8dc-48ff-a18f-e9e1de426df3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:05.565148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:05.565818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:05.566104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:05.566380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:05.579619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:05.581053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:05.596387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:05.597536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:05.599350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:05.599350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:05.599350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:05.599350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:05.599350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 637/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:05.603734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b0645-d703-4264-8c68-0aca85efbf63 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:05.728642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:05.735913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b0645-d703-4264-8c68-0aca85efbf63 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:05.773013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627343-9040-4218-9917-f61dfe60feab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:05.773770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627343-9040-4218-9917-f61dfe60feab None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:05.773961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627343-9040-4218-9917-f61dfe60feab None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:05.774308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627343-9040-4218-9917-f61dfe60feab None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:05.775621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627343-9040-4218-9917-f61dfe60feab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:05.775982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627343-9040-4218-9917-f61dfe60feab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:05.777763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627343-9040-4218-9917-f61dfe60feab None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:05.796672 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 606/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:05.798320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:05.896673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:05.924730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7bb95-b2ec-4d84-b55b-59dc600e7b09 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:05.961370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:05.962036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:05.962300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:05.962575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:05.969464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:05.971594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:27:05.989255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:05.990328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:05.991264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 638/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:27:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:05.997739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627343-9040-4218-9917-f61dfe60feab None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:06.110426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627343-9040-4218-9917-f61dfe60feab None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:06.146642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f812558-d707-4eed-9907-d8aed4b4e1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:06.147368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812558-d707-4eed-9907-d8aed4b4e1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:06.147672 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812558-d707-4eed-9907-d8aed4b4e1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:06.147966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812558-d707-4eed-9907-d8aed4b4e1d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:06.148744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f812558-d707-4eed-9907-d8aed4b4e1d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:06.149171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f812558-d707-4eed-9907-d8aed4b4e1d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:06.150799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f812558-d707-4eed-9907-d8aed4b4e1d7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:06.165258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 607/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:27:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:06.172054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:06.311748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cdb48-16bb-4c6d-b093-509ada2028cb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:06.352565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c14e21-25c6-4bae-9e38-9e243c8c1471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:06.353253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c14e21-25c6-4bae-9e38-9e243c8c1471 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:06.353587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c14e21-25c6-4bae-9e38-9e243c8c1471 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:06.353862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c14e21-25c6-4bae-9e38-9e243c8c1471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:06.356512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c14e21-25c6-4bae-9e38-9e243c8c1471 None admin] RBAC: Adding query filter params (role.id=e6fe0c4d4ed1454f935b5c9001151260, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=07b18ac45ec948738129f3d08e0d0eb3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:06.356512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c14e21-25c6-4bae-9e38-9e243c8c1471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:06.357005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c14e21-25c6-4bae-9e38-9e243c8c1471 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:06.363461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 639/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:27:06 2022] GET /identity/v3/role_assignments?user.id=07b18ac45ec948738129f3d08e0d0eb3&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=e6fe0c4d4ed1454f935b5c9001151260 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:08.676081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1460-ee89-4bcd-9b3f-a6aa491adede None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:08.676621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1460-ee89-4bcd-9b3f-a6aa491adede None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:08.676621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1460-ee89-4bcd-9b3f-a6aa491adede None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:08.678237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 608/1247] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:27:08.684613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56cf0d-2426-41b5-b725-bfb61b86316b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:08.685059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56cf0d-2426-41b5-b725-bfb61b86316b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:08.685275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56cf0d-2426-41b5-b725-bfb61b86316b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:08.692460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:08.707420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4e56cf0d-2426-41b5-b725-bfb61b86316b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:27:08.708537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4e56cf0d-2426-41b5-b725-bfb61b86316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:27:08.795421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 640/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:27:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:27:08.821174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1460-ee89-4bcd-9b3f-a6aa491adede None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:08.848105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:08.950553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1460-ee89-4bcd-9b3f-a6aa491adede None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:08.990920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138956f-2d6f-4545-a90a-44cc6b6013cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:08.991752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138956f-2d6f-4545-a90a-44cc6b6013cb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:08.992021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138956f-2d6f-4545-a90a-44cc6b6013cb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:08.992289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138956f-2d6f-4545-a90a-44cc6b6013cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:09.003731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138956f-2d6f-4545-a90a-44cc6b6013cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-3138956f-2d6f-4545-a90a-44cc6b6013cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:27:09.009066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:09.013930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:09.015014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 609/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:27:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 06:27:09.018469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56cf0d-2426-41b5-b725-bfb61b86316b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:09.143703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56cf0d-2426-41b5-b725-bfb61b86316b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:09.185538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:09.192284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:09.193575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:09.193969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:09.194290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:09.195504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:09.195935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:09.197807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:09.208338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 641/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:27:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:09.217482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138956f-2d6f-4545-a90a-44cc6b6013cb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:09.303898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:09.330637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138956f-2d6f-4545-a90a-44cc6b6013cb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:09.371012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:09.371733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:09.372013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:09.372350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:09.384209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:09.386204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:09.402238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:09.403269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:09.404579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:09.404579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:09.404579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:09.404579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:09.404579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 610/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:09.409273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:09.533724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7cdea-1d04-4c8a-bbf1-cf277fa54e3c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:09.572046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:09.572849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:09.573232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:09.573602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:09.574300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:09.574602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:09.575964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:09.588211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 642/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:27:09.594673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:09.703576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135b68-8acd-4062-9083-9f8cf69075eb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:09.733707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:09.743249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:09.744017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:09.744417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:09.744715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:09.754124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:09.755565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:27:09.770263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:09.771500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:09.772465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 611/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:09.778192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:09.844459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:09.901772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46733c-437b-44bf-af18-c71bb739bc46 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:09.940134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:09.940739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:09.941121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:09.941340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:09.942108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:09.942420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:09.943837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:09.963458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 643/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jul 28 06:27:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:09.970675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:10.075491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003222e-f2a8-44a9-b595-6ea135c5213f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:10.110470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd3cca-e0f2-46f0-b989-5d6d610cad31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:10.111016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd3cca-e0f2-46f0-b989-5d6d610cad31 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:10.111299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd3cca-e0f2-46f0-b989-5d6d610cad31 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:10.111575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd3cca-e0f2-46f0-b989-5d6d610cad31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:10.176687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:10.193887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd3cca-e0f2-46f0-b989-5d6d610cad31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:10.195420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd3cca-e0f2-46f0-b989-5d6d610cad31 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:10.305814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 612/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:27:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:27:10.314048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:10.427137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc397a8-f8e8-48d6-af70-9cfdd5dc9694 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:10.441994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:10.467004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:10.467732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:10.468187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:10.468648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:10.480504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:10.482268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:10.498006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:10.499001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:10.500111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:10.500111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:10.500111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:10.500111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:10.500111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 644/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:10.506919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd3cca-e0f2-46f0-b989-5d6d610cad31 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:10.608981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:10.623701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd3cca-e0f2-46f0-b989-5d6d610cad31 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:10.666188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:10.667382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:10.667653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:10.668027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:10.668738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:10.669046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:10.670451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:10.683605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 613/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:10.689082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:10.797002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10471765-0f1a-45bf-9c95-d9d4ab63b87c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:10.832700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:10.833350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:10.833631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:10.833897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:10.841659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:10.843090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:27:10.855340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:10.857026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:10.858619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 645/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:10.863379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:10.974849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcdb38-4c23-4ea5-a12c-132c7f1c5c4f None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:11.012319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281c86d-7ce3-433c-a238-dd49b09b17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:11.012950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281c86d-7ce3-433c-a238-dd49b09b17b3 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:11.013233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281c86d-7ce3-433c-a238-dd49b09b17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:11.013548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281c86d-7ce3-433c-a238-dd49b09b17b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:11.014317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281c86d-7ce3-433c-a238-dd49b09b17b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:11.014629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281c86d-7ce3-433c-a238-dd49b09b17b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:11.016091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281c86d-7ce3-433c-a238-dd49b09b17b3 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:11.029791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 614/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:11.036016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:11.064644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:11.151829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d090d3-9f57-47d2-9b3a-7c63cb079ef8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:11.194985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee85c8-2ff8-477a-b526-2fc2cfb4ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:11.195628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee85c8-2ff8-477a-b526-2fc2cfb4ff1e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:11.195942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee85c8-2ff8-477a-b526-2fc2cfb4ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:11.196352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee85c8-2ff8-477a-b526-2fc2cfb4ff1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:11.197439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee85c8-2ff8-477a-b526-2fc2cfb4ff1e None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=07b18ac45ec948738129f3d08e0d0eb3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:11.197793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee85c8-2ff8-477a-b526-2fc2cfb4ff1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:11.199458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee85c8-2ff8-477a-b526-2fc2cfb4ff1e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:11.206193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 646/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:27:11 2022] GET /identity/v3/role_assignments?user.id=07b18ac45ec948738129f3d08e0d0eb3&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:27:13.201018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fefbdf-cae3-49de-9588-a27957a5a649 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:13.201676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fefbdf-cae3-49de-9588-a27957a5a649 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:13.201676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fefbdf-cae3-49de-9588-a27957a5a649 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:13.202954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 615/1261] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:27:13.208121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae639f7-e409-40e7-9448-9e2414885396 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:13.208572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae639f7-e409-40e7-9448-9e2414885396 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:13.208669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae639f7-e409-40e7-9448-9e2414885396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:13.227217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dae639f7-e409-40e7-9448-9e2414885396 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:27:13.228412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dae639f7-e409-40e7-9448-9e2414885396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:27:13.293989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 647/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:27:13.317089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fefbdf-cae3-49de-9588-a27957a5a649 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:13.329195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:13.425551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fefbdf-cae3-49de-9588-a27957a5a649 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:13.460076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0923188-4f1b-4b98-a0de-4b2e5edf1dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:13.460715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0923188-4f1b-4b98-a0de-4b2e5edf1dff None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:13.460999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0923188-4f1b-4b98-a0de-4b2e5edf1dff None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:13.461273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0923188-4f1b-4b98-a0de-4b2e5edf1dff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:13.471566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0923188-4f1b-4b98-a0de-4b2e5edf1dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-a0923188-4f1b-4b98-a0de-4b2e5edf1dff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:27:13.476476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:13.477735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:13.478712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 616/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:27:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:13.484161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae639f7-e409-40e7-9448-9e2414885396 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:13.502315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:13.594599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae639f7-e409-40e7-9448-9e2414885396 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:13.630080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:13.630689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:13.630969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:13.631240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:13.631942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:13.632242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:13.633630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:13.641823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 648/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:27:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:13.647615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0923188-4f1b-4b98-a0de-4b2e5edf1dff None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:13.715319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:13.749304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0923188-4f1b-4b98-a0de-4b2e5edf1dff None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:13.782421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:13.783051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:13.783335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:13.783611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:13.794091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:13.795528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:13.810625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:13.811657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:13.812776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:13.812776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:13.812776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:13.812776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:13.812776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 617/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:13.817468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:13.901031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:13.920511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8f4e4-2664-4380-8e11-8f91afb27ff6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:13.953660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:13.954270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:13.954552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:13.954826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:13.955506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:13.955810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:13.957158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:13.968455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 649/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:13.974039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:14.070624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c109a-a8d3-4b29-b16a-a4be4da69cfb None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:14.089934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:14.103712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:14.104937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:14.105223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:14.105530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:14.112856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:14.114320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:27:14.125643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:14.126758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:14.128101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 618/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:14.132628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:14.243926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad875597-7151-47e0-bc0d-1842418277f8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:14.276576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:14.277236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:14.277537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:14.277802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:14.278570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:14.278871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:14.280278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:14.288488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:14.297375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 650/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jul 28 06:27:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:14.303117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:14.400368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3669bc-8a9a-4f21-8f77-f0793b2d6991 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:14.434080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:14.434723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:14.435006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:14.435284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:14.445817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:14.447513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:14.462505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:14.464365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:14.465569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:14.465569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:14.465569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:14.465569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:14.465569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 619/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:14.470529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:14.567572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e327c2-60cb-4310-a502-950bd3438dba None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:14.602765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:14.603467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:14.603851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:14.604334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:14.605095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:14.605568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:14.607014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:14.618130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 651/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:14.623820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:14.655804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:14.724648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e71f04-2caa-4909-b800-b412faa65583 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:14.758234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f499de-6e62-4770-9257-77df19942996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:14.758866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f499de-6e62-4770-9257-77df19942996 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:14.759147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f499de-6e62-4770-9257-77df19942996 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:14.759420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f499de-6e62-4770-9257-77df19942996 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:14.766642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f499de-6e62-4770-9257-77df19942996 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:14.768077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f499de-6e62-4770-9257-77df19942996 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-87f499de-6e62-4770-9257-77df19942996 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:27:14.779444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:14.780529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:14.781705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 620/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:27:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:14.786449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:14.844303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:14.886412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b53dfa-4200-4ce5-8f2d-cd5cd5c9b417 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:14.919201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d466d-3034-43b6-8300-3fa52fbb816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:14.919814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d466d-3034-43b6-8300-3fa52fbb816a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:14.920099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d466d-3034-43b6-8300-3fa52fbb816a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:14.920372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d466d-3034-43b6-8300-3fa52fbb816a None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:14.921113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d466d-3034-43b6-8300-3fa52fbb816a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:14.921454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d466d-3034-43b6-8300-3fa52fbb816a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:14.922911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d466d-3034-43b6-8300-3fa52fbb816a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:14.935243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 652/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:27:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:14.941224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f499de-6e62-4770-9257-77df19942996 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:15.032084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:15.040687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f499de-6e62-4770-9257-77df19942996 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:15.073785 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8349d-99a1-492c-951b-1fe31d215d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:15.074145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8349d-99a1-492c-951b-1fe31d215d58 None admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:15.074426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8349d-99a1-492c-951b-1fe31d215d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:15.074696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8349d-99a1-492c-951b-1fe31d215d58 None admin] PATH_INFO: `/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/07b18ac45ec948738129f3d08e0d0eb3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:15.076581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8349d-99a1-492c-951b-1fe31d215d58 None admin] RBAC: Authorizing `identity:create_grant(project_id=473e18bb7e8542bbb3c23b7d813c583e, user_id=07b18ac45ec948738129f3d08e0d0eb3, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:15.078034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8349d-99a1-492c-951b-1fe31d215d58 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:15.089737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 621/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Jul 28 06:27:14 2022] PUT /identity/v3/projects/473e18bb7e8542bbb3c23b7d813c583e/users/07b18ac45ec948738129f3d08e0d0eb3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:27:17.114949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714388-febe-4eb2-95c0-de77d7f73cd3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:17.115577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714388-febe-4eb2-95c0-de77d7f73cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:17.115577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714388-febe-4eb2-95c0-de77d7f73cd3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:17.116883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 653/1274] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:27:17.122384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a040e21-1c11-4134-90a0-4ee7937b9a4f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:17.122827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a040e21-1c11-4134-90a0-4ee7937b9a4f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:17.122931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a040e21-1c11-4134-90a0-4ee7937b9a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:17.140720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4a040e21-1c11-4134-90a0-4ee7937b9a4f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:27:17.141923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4a040e21-1c11-4134-90a0-4ee7937b9a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:27:17.218847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 622/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:27:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:27:17.242604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714388-febe-4eb2-95c0-de77d7f73cd3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:17.343386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714388-febe-4eb2-95c0-de77d7f73cd3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:17.360439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:17.378201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c34a71-efcd-45a5-bf88-41de03c1501e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:17.378770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c34a71-efcd-45a5-bf88-41de03c1501e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:17.379053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c34a71-efcd-45a5-bf88-41de03c1501e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:17.379328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c34a71-efcd-45a5-bf88-41de03c1501e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:17.389454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c34a71-efcd-45a5-bf88-41de03c1501e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-20c34a71-efcd-45a5-bf88-41de03c1501e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 06:27:17.394280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:17.395474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:17.396742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 654/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Jul 28 06:27:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:17.401517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a040e21-1c11-4134-90a0-4ee7937b9a4f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:17.482405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:17.502206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a040e21-1c11-4134-90a0-4ee7937b9a4f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:17.535708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:17.536322 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:17.536604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:17.536878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:17.537618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:17.537922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:17.539281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:17.548326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 623/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Jul 28 06:27:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:17.554084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c34a71-efcd-45a5-bf88-41de03c1501e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:17.653033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c34a71-efcd-45a5-bf88-41de03c1501e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:17.686649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:17.687282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:17.687563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:17.687837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:17.697927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:17.699342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:17.713821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:17.714897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:17.716054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:17.716054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:17.716054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:17.716054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:17.716054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 655/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:17.720439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:17.822839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be965-492a-4d5d-b4e4-e46772c64d25 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:17.864680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:17.865308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:17.865604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:17.865879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:17.866568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:17.866872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:17.868238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:17.879695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 624/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:17.885424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:17.915959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:17.983870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c819be1-c3f7-49da-8b74-15413637e373 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:18.015261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:18.015884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:18.016160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:18.016427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:18.023485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:18.024886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 06:27:18.036129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:18.037318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:18.038340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 656/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:18.043676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:18.047430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:18.142701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1b35a-a314-498f-ae5e-10c0fed53ddd None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:18.187946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:18.188486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:18.188766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:18.189036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:18.189841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:18.190143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:18.191551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:18.208923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 625/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Jul 28 06:27:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:18.215376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:18.303059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:18.313424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7414-4423-42b3-a9a3-8a5daa60b004 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:18.347092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42184df6-e33d-430d-96bf-d758e345075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:18.347698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42184df6-e33d-430d-96bf-d758e345075e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:18.347982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42184df6-e33d-430d-96bf-d758e345075e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:18.348255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42184df6-e33d-430d-96bf-d758e345075e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:18.413561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42184df6-e33d-430d-96bf-d758e345075e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:18.415002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42184df6-e33d-430d-96bf-d758e345075e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:18.503856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:18.515073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 657/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Jul 28 06:27:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 06:27:18.520962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:18.581580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:18.618903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66cf05-cff1-4d55-9ece-8346cdd83dec None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:18.653312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:18.653959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:18.654243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:18.654516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:18.664325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:18.665813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 06:27:18.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:18.681198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:18.682762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 06:27:18.682762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 06:27:18.682762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 06:27:18.682762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:18.682762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 626/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:18.687667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42184df6-e33d-430d-96bf-d758e345075e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:18.785202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42184df6-e33d-430d-96bf-d758e345075e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:18.818527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:18.819139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:18.819422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:18.819697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:18.820383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:18.820691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:18.822325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:18.833878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 658/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Jul 28 06:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:18.839814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:18.940126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63132af4-0ef9-4eb2-b357-00e88b82515c None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:18.953910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:18.973645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:18.974275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:18.974555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:18.974829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:18.982257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:18.983683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 06:27:18.995323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:18.996447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:18.997626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 06:27:18.997626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 06:27:18.997626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 06:27:18.997626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 06:27:18.997626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 06:27:18.997626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 06:27:18.997626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 06:27:18.997626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: ERROR keystone.server.flask.application Jul 28 06:27:19.001225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 627/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Jul 28 06:27:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:19.002377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:19.050862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:19.100012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33a434-d6df-43f0-85d3-872dd135f5c8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:19.132648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61418e7-766f-42c8-8312-2c3218247b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:19.133254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61418e7-766f-42c8-8312-2c3218247b14 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:19.133563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61418e7-766f-42c8-8312-2c3218247b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:19.133841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61418e7-766f-42c8-8312-2c3218247b14 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:19.134576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61418e7-766f-42c8-8312-2c3218247b14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:19.134882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61418e7-766f-42c8-8312-2c3218247b14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:19.136433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61418e7-766f-42c8-8312-2c3218247b14 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:19.147886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 659/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Jul 28 06:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:19.155972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:19.259006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba9722-918d-46a2-880a-c315db05a06b None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:19.292691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb400f6-0344-4ca2-b2b0-228c29060158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:19.293319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb400f6-0344-4ca2-b2b0-228c29060158 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:19.293611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb400f6-0344-4ca2-b2b0-228c29060158 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:19.293887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb400f6-0344-4ca2-b2b0-228c29060158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:19.294666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb400f6-0344-4ca2-b2b0-228c29060158 None admin] RBAC: Adding query filter params (role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c, scope.project.id=473e18bb7e8542bbb3c23b7d813c583e, user.id=07b18ac45ec948738129f3d08e0d0eb3) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:19.294977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb400f6-0344-4ca2-b2b0-228c29060158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:19.296393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb400f6-0344-4ca2-b2b0-228c29060158 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:19.302090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 628/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Jul 28 06:27:19 2022] GET /identity/v3/role_assignments?user.id=07b18ac45ec948738129f3d08e0d0eb3&scope.project.id=473e18bb7e8542bbb3c23b7d813c583e&role.id=ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:27:21.313845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cf80b-9f40-4305-9138-03b96eadd13c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:21.314407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cf80b-9f40-4305-9138-03b96eadd13c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:21.314407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cf80b-9f40-4305-9138-03b96eadd13c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:21.315724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 660/1288] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:27:21.320725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bf93a-6e9f-4ccd-bcea-21446d76d830 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:21.321113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bf93a-6e9f-4ccd-bcea-21446d76d830 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:21.321306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bf93a-6e9f-4ccd-bcea-21446d76d830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:21.339815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f57bf93a-6e9f-4ccd-bcea-21446d76d830 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:27:21.340999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f57bf93a-6e9f-4ccd-bcea-21446d76d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:27:21.370900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:21.406912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 629/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:27:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:27:21.429845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cf80b-9f40-4305-9138-03b96eadd13c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:21.531750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cf80b-9f40-4305-9138-03b96eadd13c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:21.542770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:21.565880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:21.566455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:21.566739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:21.567016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:21.567641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:21.568999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:21.577934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:21.579248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:21.580427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 661/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:27:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:21.585116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bf93a-6e9f-4ccd-bcea-21446d76d830 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:21.683031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bf93a-6e9f-4ccd-bcea-21446d76d830 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:21.716363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67795b24-62ba-45f9-8bab-48a67981bee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:21.716979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67795b24-62ba-45f9-8bab-48a67981bee0 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:21.717264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67795b24-62ba-45f9-8bab-48a67981bee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:21.717574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67795b24-62ba-45f9-8bab-48a67981bee0 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:21.718249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67795b24-62ba-45f9-8bab-48a67981bee0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:21.718556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67795b24-62ba-45f9-8bab-48a67981bee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:21.719919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67795b24-62ba-45f9-8bab-48a67981bee0 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:21.724613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 630/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Jul 28 06:27:21 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:21.730466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:21.828083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd659d0-15f5-4dd2-acd0-a70f8760f9e9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:21.862158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec801c42-da12-4321-a22a-dd11f41404ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:21.862767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec801c42-da12-4321-a22a-dd11f41404ea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:21.863051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec801c42-da12-4321-a22a-dd11f41404ea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:21.863327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec801c42-da12-4321-a22a-dd11f41404ea None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:21.864002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec801c42-da12-4321-a22a-dd11f41404ea None admin] RBAC: Adding query filter params (type=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:21.864308 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec801c42-da12-4321-a22a-dd11f41404ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:21.865686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec801c42-da12-4321-a22a-dd11f41404ea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:21.870099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 662/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Jul 28 06:27:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:23.897266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be549e8d-a420-4cf2-b65d-ea8fdcb6f208 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:23.897874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be549e8d-a420-4cf2-b65d-ea8fdcb6f208 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:23.897874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be549e8d-a420-4cf2-b65d-ea8fdcb6f208 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:23.899210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 631/1293] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:27:23.905417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fddd2-1e9c-41c5-b577-9da28b5b71ff None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:23.905889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fddd2-1e9c-41c5-b577-9da28b5b71ff None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:23.906133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fddd2-1e9c-41c5-b577-9da28b5b71ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:23.928333 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e70fddd2-1e9c-41c5-b577-9da28b5b71ff None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:27:23.929599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e70fddd2-1e9c-41c5-b577-9da28b5b71ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:27:23.960391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:24.000182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 663/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:27:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 06:27:24.023691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be549e8d-a420-4cf2-b65d-ea8fdcb6f208 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:24.055263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:24.130653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be549e8d-a420-4cf2-b65d-ea8fdcb6f208 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:24.172565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88eedee-b25a-4958-bec3-020fd680fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:24.173332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eedee-b25a-4958-bec3-020fd680fc6f None admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:24.173661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eedee-b25a-4958-bec3-020fd680fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:24.173960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eedee-b25a-4958-bec3-020fd680fc6f None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:24.174926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88eedee-b25a-4958-bec3-020fd680fc6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:24.176873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88eedee-b25a-4958-bec3-020fd680fc6f None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:24.198823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 632/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Jul 28 06:27:24 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:27:26.247677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f5786-bb9f-4fd1-9e7e-5c2e5519a9bd None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:26.248216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f5786-bb9f-4fd1-9e7e-5c2e5519a9bd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:26.248216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f5786-bb9f-4fd1-9e7e-5c2e5519a9bd None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:26.249640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 664/1296] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:27:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:27:26.254720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb33ce9-8651-41d4-9d25-c3d18784ee3b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:26.255186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb33ce9-8651-41d4-9d25-c3d18784ee3b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:26.255296 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb33ce9-8651-41d4-9d25-c3d18784ee3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:26.273602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5eb33ce9-8651-41d4-9d25-c3d18784ee3b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:27:26.274796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5eb33ce9-8651-41d4-9d25-c3d18784ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:27:26.346828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 633/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:27:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:27:26.369214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f5786-bb9f-4fd1-9e7e-5c2e5519a9bd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:26.470112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f5786-bb9f-4fd1-9e7e-5c2e5519a9bd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:26.503475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:26.504036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:26.504312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:26.504580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:26.505193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:26.506561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:26.515176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:26.516367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:26.517467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 665/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:27:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:26.522392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb33ce9-8651-41d4-9d25-c3d18784ee3b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:26.540243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:26.625986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb33ce9-8651-41d4-9d25-c3d18784ee3b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:26.658690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:26.659311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:26.659596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:26.659874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:26.660547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:26.660850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:26.662245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:26.667767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 634/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Jul 28 06:27:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:26.673975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:26.705492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:26.772636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043cf0c-32f0-4ac5-a652-50905a6f686f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:26.807866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ad808-a464-4b3a-be43-eb129f80e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:26.808494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad808-a464-4b3a-be43-eb129f80e450 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:26.808779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad808-a464-4b3a-be43-eb129f80e450 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:26.809055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad808-a464-4b3a-be43-eb129f80e450 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:26.809843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ad808-a464-4b3a-be43-eb129f80e450 None admin] RBAC: Adding query filter params (interface=public, service_id=27c0303dcb18487aab53f2c71f4c08cb, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:26.810157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ad808-a464-4b3a-be43-eb129f80e450 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:26.811514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ad808-a464-4b3a-be43-eb129f80e450 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:26.816479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 666/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Jul 28 06:27:26 2022] GET /identity/v3/endpoints?service_id=27c0303dcb18487aab53f2c71f4c08cb&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:26.822421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:26.902584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:26.921106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500055a-c7c7-4954-a4e0-9fd3e3aba7f9 None admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:26.954696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab78bcb-8121-40ab-b95b-529bc830f9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:26.955309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab78bcb-8121-40ab-b95b-529bc830f9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:26.955590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab78bcb-8121-40ab-b95b-529bc830f9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:26.955866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab78bcb-8121-40ab-b95b-529bc830f9f7 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:26.956513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab78bcb-8121-40ab-b95b-529bc830f9f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:26.957907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab78bcb-8121-40ab-b95b-529bc830f9f7 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:26.965016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 635/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Jul 28 06:27:26 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:27:28.962702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d5cd3-f2da-416f-bff7-d17511387502 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:28.963304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d5cd3-f2da-416f-bff7-d17511387502 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:28.963840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d5cd3-f2da-416f-bff7-d17511387502 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:28.965197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 667/1302] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:27:28.970307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f062-c659-4b99-ad83-d4cc1b9b4911 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:28.970708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f062-c659-4b99-ad83-d4cc1b9b4911 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:28.970863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f062-c659-4b99-ad83-d4cc1b9b4911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:28.989136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cc73f062-c659-4b99-ad83-d4cc1b9b4911 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:27:28.990352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cc73f062-c659-4b99-ad83-d4cc1b9b4911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:27:29.055339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 636/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:27:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 06:27:29.078239 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d5cd3-f2da-416f-bff7-d17511387502 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:29.175856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:29.184454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d5cd3-f2da-416f-bff7-d17511387502 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:29.223002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:29.223639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:29.223917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:29.224187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:29.224810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:29.226186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 06:27:29.235439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 06:27:29.236684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Jul 28 06:27:29.237631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 668/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Jul 28 06:27:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 06:27:29.242733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73f062-c659-4b99-ad83-d4cc1b9b4911 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:29.318669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:29.341613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73f062-c659-4b99-ad83-d4cc1b9b4911 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:29.374705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a56d83-710b-4135-b3d8-3ebebb580655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:29.375309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56d83-710b-4135-b3d8-3ebebb580655 None admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:29.375588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56d83-710b-4135-b3d8-3ebebb580655 None admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:29.375859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56d83-710b-4135-b3d8-3ebebb580655 None admin] PATH_INFO: `/v3/services` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:29.376526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a56d83-710b-4135-b3d8-3ebebb580655 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:27:29.376826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a56d83-710b-4135-b3d8-3ebebb580655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:29.378199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a56d83-710b-4135-b3d8-3ebebb580655 None admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:29.383137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 637/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Jul 28 06:27:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:27:29.389573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:27:29.486920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22d1d-ec37-4544-bc3a-7aed1269e306 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:27:29.519028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83644288-4de3-42dc-809e-0ef23bfa5264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:27:29.519633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83644288-4de3-42dc-809e-0ef23bfa5264 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:27:29.519914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83644288-4de3-42dc-809e-0ef23bfa5264 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:27:29.520187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83644288-4de3-42dc-809e-0ef23bfa5264 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:27:29.520882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83644288-4de3-42dc-809e-0ef23bfa5264 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:27:29.522266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83644288-4de3-42dc-809e-0ef23bfa5264 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:27:29.544025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 669/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:27:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:27:39.380276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=30 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:27:39.539448 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:33.192133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7505-7d62-438e-b77f-bbb411f560c3 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.193157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7505-7d62-438e-b77f-bbb411f560c3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.193747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7505-7d62-438e-b77f-bbb411f560c3 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.201684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f844-75d4-4b15-8ef4-f762a9acaf9c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.202572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f844-75d4-4b15-8ef4-f762a9acaf9c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.203062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f844-75d4-4b15-8ef4-f762a9acaf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.208255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 638/1307] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:33 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:28:33.284757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9b87f844-75d4-4b15-8ef4-f762a9acaf9c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:33.286361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b87f844-75d4-4b15-8ef4-f762a9acaf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:33.515447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 670/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:33.547120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09f5f4-0935-4172-90ba-34c72be3d25b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.547851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09f5f4-0935-4172-90ba-34c72be3d25b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.548356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09f5f4-0935-4172-90ba-34c72be3d25b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.550925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 639/1309] 10.222.0.20 () {54 vars in 965 bytes} [Thu Jul 28 06:28:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:33.559113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3313f-0595-4c01-8831-0a8c8f4dc69f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:33.559113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3313f-0595-4c01-8831-0a8c8f4dc69f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:33.559381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3313f-0595-4c01-8831-0a8c8f4dc69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:33.654409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-07e3313f-0595-4c01-8831-0a8c8f4dc69f None None] MFA Rules not processed for user `4b46a15d481f4393bef8c4fa2281710b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:33.656087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-07e3313f-0595-4c01-8831-0a8c8f4dc69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:33.690544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:33.837689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 671/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Thu Jul 28 06:28:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:33.838398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09f5f4-0935-4172-90ba-34c72be3d25b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:33.840158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cb09f5f4-0935-4172-90ba-34c72be3d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:33.990180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09f5f4-0935-4172-90ba-34c72be3d25b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:34.081549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3bef0-49a0-4d8a-84a1-a0de5a3233d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:34.082178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3bef0-49a0-4d8a-84a1-a0de5a3233d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:34.082464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3bef0-49a0-4d8a-84a1-a0de5a3233d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:34.082740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3bef0-49a0-4d8a-84a1-a0de5a3233d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:34.084488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-09c3bef0-49a0-4d8a-84a1-a0de5a3233d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:34.145245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:34.207378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3bef0-49a0-4d8a-84a1-a0de5a3233d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:34.208997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3bef0-49a0-4d8a-84a1-a0de5a3233d4 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:34.232076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 640/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:36.579753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04e30e-a775-4a9f-80da-b5da85925f23 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:36.580303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04e30e-a775-4a9f-80da-b5da85925f23 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:36.580303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04e30e-a775-4a9f-80da-b5da85925f23 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:36.581701 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 672/1312] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:36.587468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d7fbd-0323-4159-ba3e-4096c7c72a1d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:36.587913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d7fbd-0323-4159-ba3e-4096c7c72a1d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:36.588011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d7fbd-0323-4159-ba3e-4096c7c72a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:36.669599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-846d7fbd-0323-4159-ba3e-4096c7c72a1d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:36.670823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-846d7fbd-0323-4159-ba3e-4096c7c72a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:36.769657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 641/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:36.808994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04e30e-a775-4a9f-80da-b5da85925f23 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:36.810600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1c04e30e-a775-4a9f-80da-b5da85925f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:36.916479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04e30e-a775-4a9f-80da-b5da85925f23 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:36.992612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:36.995806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e91d4-1b26-4c23-975c-e0b4a26dac61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:36.996851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e91d4-1b26-4c23-975c-e0b4a26dac61 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:36.997426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e91d4-1b26-4c23-975c-e0b4a26dac61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:36.997965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e91d4-1b26-4c23-975c-e0b4a26dac61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:36.999715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b77e91d4-1b26-4c23-975c-e0b4a26dac61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:37.125657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e91d4-1b26-4c23-975c-e0b4a26dac61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:37.127037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e91d4-1b26-4c23-975c-e0b4a26dac61 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:37.146543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 673/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:39.518106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8acea5-9e5b-47cf-82a9-abeb60160ca6 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.518681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8acea5-9e5b-47cf-82a9-abeb60160ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.518681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8acea5-9e5b-47cf-82a9-abeb60160ca6 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.520014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 642/1315] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:28:39.525749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79691e6-50e7-4bae-a40c-38f25c748c73 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.526142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79691e6-50e7-4bae-a40c-38f25c748c73 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.526306 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79691e6-50e7-4bae-a40c-38f25c748c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.590824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d79691e6-50e7-4bae-a40c-38f25c748c73 None None] MFA Rules not processed for user `49483f572c9c483b9311fe1b812f3c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:39.592022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d79691e6-50e7-4bae-a40c-38f25c748c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:39.643885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:39.689253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 674/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:39 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:39.744629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853d480-fb9a-4cc3-85c7-9b5a806b1cce None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.744917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853d480-fb9a-4cc3-85c7-9b5a806b1cce None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.745197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853d480-fb9a-4cc3-85c7-9b5a806b1cce None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.746585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 643/1317] 10.222.0.20 () {54 vars in 965 bytes} [Thu Jul 28 06:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:39.752215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0d8bb-ed0e-4dc9-a6cc-2f4f54de4a68 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:39.752504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0d8bb-ed0e-4dc9-a6cc-2f4f54de4a68 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:39.752777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0d8bb-ed0e-4dc9-a6cc-2f4f54de4a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:39.827968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-42e0d8bb-ed0e-4dc9-a6cc-2f4f54de4a68 None None] MFA Rules not processed for user `4b46a15d481f4393bef8c4fa2281710b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:39.829178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-42e0d8bb-ed0e-4dc9-a6cc-2f4f54de4a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:39.909151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 675/1318] 10.222.0.20 () {58 vars in 1062 bytes} [Thu Jul 28 06:28:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:39.918206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853d480-fb9a-4cc3-85c7-9b5a806b1cce None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:39.919742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b853d480-fb9a-4cc3-85c7-9b5a806b1cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:39.929408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:40.025691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853d480-fb9a-4cc3-85c7-9b5a806b1cce None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:40.097648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3821e-e7a2-4a5e-8d19-1a1e4f2c1342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:40.098257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3821e-e7a2-4a5e-8d19-1a1e4f2c1342 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:40.098536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3821e-e7a2-4a5e-8d19-1a1e4f2c1342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:40.098808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3821e-e7a2-4a5e-8d19-1a1e4f2c1342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:40.100427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c1b3821e-e7a2-4a5e-8d19-1a1e4f2c1342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:40.161897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:40.209442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3821e-e7a2-4a5e-8d19-1a1e4f2c1342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:40.211512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3821e-e7a2-4a5e-8d19-1a1e4f2c1342 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:40.229200 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 644/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:43.172885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfd3af-a3b8-4e8d-a58f-7241d9aac309 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:43.173440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfd3af-a3b8-4e8d-a58f-7241d9aac309 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:43.173569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfd3af-a3b8-4e8d-a58f-7241d9aac309 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:43.174844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 676/1320] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:43.180870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468162e-875c-4cff-b8ec-23cb899d2845 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:43.181281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468162e-875c-4cff-b8ec-23cb899d2845 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:43.181457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468162e-875c-4cff-b8ec-23cb899d2845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:43.245861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f468162e-875c-4cff-b8ec-23cb899d2845 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:43.247052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f468162e-875c-4cff-b8ec-23cb899d2845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:43.341436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 645/1321] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:43.379723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfd3af-a3b8-4e8d-a58f-7241d9aac309 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:43.381317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-89cfd3af-a3b8-4e8d-a58f-7241d9aac309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:43.430894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:43.489709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfd3af-a3b8-4e8d-a58f-7241d9aac309 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:43.563199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb68b-68c1-46c6-9428-b2b52a6f1711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:43.563814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb68b-68c1-46c6-9428-b2b52a6f1711 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:43.564096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb68b-68c1-46c6-9428-b2b52a6f1711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:43.564368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb68b-68c1-46c6-9428-b2b52a6f1711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:43.566014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f9edb68b-68c1-46c6-9428-b2b52a6f1711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:43.662188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:43.675819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb68b-68c1-46c6-9428-b2b52a6f1711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:43.677249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb68b-68c1-46c6-9428-b2b52a6f1711 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:43.696685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 677/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:46.453597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6e2d3-0e77-4fc3-80c2-2236cd3a2ecc None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.454039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6e2d3-0e77-4fc3-80c2-2236cd3a2ecc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.454169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6e2d3-0e77-4fc3-80c2-2236cd3a2ecc None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.455540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 646/1323] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:46.460690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37397b-e6b6-44d5-9b97-a984f9a13cbe None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.461111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37397b-e6b6-44d5-9b97-a984f9a13cbe None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.461279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37397b-e6b6-44d5-9b97-a984f9a13cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.530368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9f37397b-e6b6-44d5-9b97-a984f9a13cbe None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:46.531581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9f37397b-e6b6-44d5-9b97-a984f9a13cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:46.635082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 678/1324] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:46.672706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6e2d3-0e77-4fc3-80c2-2236cd3a2ecc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:46.674301 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ffe6e2d3-0e77-4fc3-80c2-2236cd3a2ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:46.698516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:46.795874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6e2d3-0e77-4fc3-80c2-2236cd3a2ecc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:46.876010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87053a-337b-4566-bcc5-e874754f7d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:46.876627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87053a-337b-4566-bcc5-e874754f7d21 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:46.876908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87053a-337b-4566-bcc5-e874754f7d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:46.877180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87053a-337b-4566-bcc5-e874754f7d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:46.878814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4c87053a-337b-4566-bcc5-e874754f7d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:46.955056 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:46.995678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87053a-337b-4566-bcc5-e874754f7d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:46.997943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87053a-337b-4566-bcc5-e874754f7d21 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:47.020087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 647/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:49.098474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9e27-c2b4-4632-a24c-607f43ad976f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:49.099049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9e27-c2b4-4632-a24c-607f43ad976f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:49.099049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9e27-c2b4-4632-a24c-607f43ad976f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:49.100418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 679/1326] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:49.106250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b141395-5dd2-422d-8526-c9469a97f892 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:49.106665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b141395-5dd2-422d-8526-c9469a97f892 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:49.106804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b141395-5dd2-422d-8526-c9469a97f892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:49.186992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5b141395-5dd2-422d-8526-c9469a97f892 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:49.188129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5b141395-5dd2-422d-8526-c9469a97f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:49.292279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 648/1327] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:49.330703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed9e27-c2b4-4632-a24c-607f43ad976f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:49.332334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-53ed9e27-c2b4-4632-a24c-607f43ad976f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:49.393419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:49.439485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed9e27-c2b4-4632-a24c-607f43ad976f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:49.518340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a39d4-e2e7-49cb-970c-5719fd4040cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:49.518963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a39d4-e2e7-49cb-970c-5719fd4040cd service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:49.519266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a39d4-e2e7-49cb-970c-5719fd4040cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:49.519540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a39d4-e2e7-49cb-970c-5719fd4040cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:49.521159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-095a39d4-e2e7-49cb-970c-5719fd4040cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:49.632613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:49.635500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a39d4-e2e7-49cb-970c-5719fd4040cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:49.637615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a39d4-e2e7-49cb-970c-5719fd4040cd service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:49.655758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 680/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:51.773523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf37a-5e8b-40d4-9535-4bcec6e3cc23 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:51.774086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf37a-5e8b-40d4-9535-4bcec6e3cc23 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:51.774086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf37a-5e8b-40d4-9535-4bcec6e3cc23 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:51.775431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 649/1329] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:51.780751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde2646-2ae6-4bd8-b484-5d57d53d6938 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:51.781175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde2646-2ae6-4bd8-b484-5d57d53d6938 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:51.781287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde2646-2ae6-4bd8-b484-5d57d53d6938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:51.847314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dcde2646-2ae6-4bd8-b484-5d57d53d6938 None None] MFA Rules not processed for user `49483f572c9c483b9311fe1b812f3c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:51.848503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dcde2646-2ae6-4bd8-b484-5d57d53d6938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:51.947109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 681/1330] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:51 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:51.986105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbf37a-5e8b-40d4-9535-4bcec6e3cc23 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:51.987791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-71cbf37a-5e8b-40d4-9535-4bcec6e3cc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:52.020186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:52.094141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbf37a-5e8b-40d4-9535-4bcec6e3cc23 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:52.169518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfeb0d-5327-4149-a9af-ca6b84ddb92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:52.170207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfeb0d-5327-4149-a9af-ca6b84ddb92a service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:52.170489 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfeb0d-5327-4149-a9af-ca6b84ddb92a service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:52.170767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfeb0d-5327-4149-a9af-ca6b84ddb92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:52.172399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-76cfeb0d-5327-4149-a9af-ca6b84ddb92a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:52.253789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:52.285234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfeb0d-5327-4149-a9af-ca6b84ddb92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:52.286723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfeb0d-5327-4149-a9af-ca6b84ddb92a service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:52.306067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 650/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:51 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:54.635666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c7842-f3c8-455f-bcc9-ed48f16f3111 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:54.636213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c7842-f3c8-455f-bcc9-ed48f16f3111 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:54.636213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c7842-f3c8-455f-bcc9-ed48f16f3111 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:54.637625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 682/1332] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:54.642844 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6583c1-6622-46f7-823b-ab0af05f4167 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:54.643250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6583c1-6622-46f7-823b-ab0af05f4167 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:54.643396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6583c1-6622-46f7-823b-ab0af05f4167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:54.709347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ff6583c1-6622-46f7-823b-ab0af05f4167 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:54.710762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ff6583c1-6622-46f7-823b-ab0af05f4167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:54.805451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 651/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:28:54.842229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c7842-f3c8-455f-bcc9-ed48f16f3111 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:54.843764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a14c7842-f3c8-455f-bcc9-ed48f16f3111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:54.914330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:54.948568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c7842-f3c8-455f-bcc9-ed48f16f3111 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:55.020968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecb485-aec4-4fb6-a864-f574a3b67d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:55.021610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecb485-aec4-4fb6-a864-f574a3b67d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:55.021892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecb485-aec4-4fb6-a864-f574a3b67d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:55.022175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecb485-aec4-4fb6-a864-f574a3b67d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:55.024577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-01ecb485-aec4-4fb6-a864-f574a3b67d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:55.132605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecb485-aec4-4fb6-a864-f574a3b67d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:55.134090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecb485-aec4-4fb6-a864-f574a3b67d9d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:55.144145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:55.152674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 683/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:28:57.199055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65445f4-be4a-4c63-b29c-345f97f9fbd3 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:57.199641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65445f4-be4a-4c63-b29c-345f97f9fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:57.199641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65445f4-be4a-4c63-b29c-345f97f9fbd3 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:57.200964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 652/1335] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:28:57.206443 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8dde-2a5a-42ca-a56c-336ec5cfd8f7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:57.206866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8dde-2a5a-42ca-a56c-336ec5cfd8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:57.207006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8dde-2a5a-42ca-a56c-336ec5cfd8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:57.277555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6f8d8dde-2a5a-42ca-a56c-336ec5cfd8f7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:28:57.278768 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6f8d8dde-2a5a-42ca-a56c-336ec5cfd8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:57.374269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 684/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:28:57.415520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65445f4-be4a-4c63-b29c-345f97f9fbd3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:28:57.417028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c65445f4-be4a-4c63-b29c-345f97f9fbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:57.457835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:57.520707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65445f4-be4a-4c63-b29c-345f97f9fbd3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:28:57.596374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b86f6-8560-4bb2-8946-f76a1fd1b179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:28:57.597068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b86f6-8560-4bb2-8946-f76a1fd1b179 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:28:57.597388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b86f6-8560-4bb2-8946-f76a1fd1b179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:28:57.597667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b86f6-8560-4bb2-8946-f76a1fd1b179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:28:57.599294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8d4b86f6-8560-4bb2-8946-f76a1fd1b179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:28:57.682988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:28:57.708830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b86f6-8560-4bb2-8946-f76a1fd1b179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:28:57.710752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b86f6-8560-4bb2-8946-f76a1fd1b179 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:28:57.729023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 653/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:00.120424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2b31c-5399-4e30-8f44-11ca6a6f90af None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:00.120983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2b31c-5399-4e30-8f44-11ca6a6f90af None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:00.120983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2b31c-5399-4e30-8f44-11ca6a6f90af None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:00.122385 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 685/1338] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:00.127519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546af434-bf2a-4f44-9d20-26862576441e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:00.127929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546af434-bf2a-4f44-9d20-26862576441e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:00.128072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546af434-bf2a-4f44-9d20-26862576441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:00.194466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-546af434-bf2a-4f44-9d20-26862576441e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:00.195666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-546af434-bf2a-4f44-9d20-26862576441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:00.288786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 654/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:00.325014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2b31c-5399-4e30-8f44-11ca6a6f90af None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:00.326578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dce2b31c-5399-4e30-8f44-11ca6a6f90af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:00.402543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:00.431975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2b31c-5399-4e30-8f44-11ca6a6f90af None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:00.505688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e1321-2ae6-42ad-a026-def64143b7b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:00.506307 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1321-2ae6-42ad-a026-def64143b7b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:00.506591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1321-2ae6-42ad-a026-def64143b7b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:00.506865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1321-2ae6-42ad-a026-def64143b7b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:00.508489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-824e1321-2ae6-42ad-a026-def64143b7b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:00.617316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e1321-2ae6-42ad-a026-def64143b7b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:00.618669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e1321-2ae6-42ad-a026-def64143b7b0 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:00.638105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 686/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:04.585196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45076cb-fc0d-4a1d-b040-71bfcf0941f2 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:04.585734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45076cb-fc0d-4a1d-b040-71bfcf0941f2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:04.585831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45076cb-fc0d-4a1d-b040-71bfcf0941f2 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:04.587138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 655/1341] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:04.592345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc159-5e5a-4a26-b1ed-227e64898480 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:04.592786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc159-5e5a-4a26-b1ed-227e64898480 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:04.592935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc159-5e5a-4a26-b1ed-227e64898480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:04.598636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:04.661589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cd0fc159-5e5a-4a26-b1ed-227e64898480 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:04.663332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cd0fc159-5e5a-4a26-b1ed-227e64898480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:04.765904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 687/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:04.806058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45076cb-fc0d-4a1d-b040-71bfcf0941f2 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:04.809060 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e45076cb-fc0d-4a1d-b040-71bfcf0941f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:04.879592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:04.935882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45076cb-fc0d-4a1d-b040-71bfcf0941f2 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:05.010459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cc134-2779-4eb1-87a1-530f03d47415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:05.011080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc134-2779-4eb1-87a1-530f03d47415 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:05.011366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc134-2779-4eb1-87a1-530f03d47415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:05.011643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cc134-2779-4eb1-87a1-530f03d47415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:05.013255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9f8cc134-2779-4eb1-87a1-530f03d47415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:05.106443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:05.123459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cc134-2779-4eb1-87a1-530f03d47415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:05.124897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cc134-2779-4eb1-87a1-530f03d47415 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:05.143015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 656/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:07.883281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981b83-1603-4213-a223-c4ad7d15b59c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:07.883837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981b83-1603-4213-a223-c4ad7d15b59c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:07.883837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981b83-1603-4213-a223-c4ad7d15b59c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:07.885274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 688/1344] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:07.891139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a6848-56d6-4802-9660-19b61ab54a57 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:07.891679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a6848-56d6-4802-9660-19b61ab54a57 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:07.891679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a6848-56d6-4802-9660-19b61ab54a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:07.963641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-136a6848-56d6-4802-9660-19b61ab54a57 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:07.964849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-136a6848-56d6-4802-9660-19b61ab54a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:08.065188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 657/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:08.103767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a981b83-1603-4213-a223-c4ad7d15b59c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:08.105307 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7a981b83-1603-4213-a223-c4ad7d15b59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:08.206541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:08.224800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a981b83-1603-4213-a223-c4ad7d15b59c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:08.300148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bb412-aba1-425f-89bb-cb2b0dafe6c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:08.300761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bb412-aba1-425f-89bb-cb2b0dafe6c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:08.301034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bb412-aba1-425f-89bb-cb2b0dafe6c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:08.301315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bb412-aba1-425f-89bb-cb2b0dafe6c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:08.302935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4d6bb412-aba1-425f-89bb-cb2b0dafe6c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:08.414862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bb412-aba1-425f-89bb-cb2b0dafe6c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:08.416292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bb412-aba1-425f-89bb-cb2b0dafe6c6 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:08.437835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 689/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:12.457170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf5b0-6738-4fac-bd38-24dba95e2e54 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.457634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf5b0-6738-4fac-bd38-24dba95e2e54 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.457748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf5b0-6738-4fac-bd38-24dba95e2e54 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.459160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 658/1347] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:12.464982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edae5e-dc7f-46e0-9cd0-102fff3247c7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.465480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edae5e-dc7f-46e0-9cd0-102fff3247c7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.465654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edae5e-dc7f-46e0-9cd0-102fff3247c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.482107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:12.539534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-82edae5e-dc7f-46e0-9cd0-102fff3247c7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:12.540904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-82edae5e-dc7f-46e0-9cd0-102fff3247c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:12.671127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 690/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:12.711246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf5b0-6738-4fac-bd38-24dba95e2e54 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:12.712772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-663cf5b0-6738-4fac-bd38-24dba95e2e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:12.774516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:12.823113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf5b0-6738-4fac-bd38-24dba95e2e54 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:12.901828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1165ae-b29b-4917-be9b-f9ca42674988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:12.902459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1165ae-b29b-4917-be9b-f9ca42674988 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:12.902742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1165ae-b29b-4917-be9b-f9ca42674988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:12.903018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1165ae-b29b-4917-be9b-f9ca42674988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:12.904670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ab1165ae-b29b-4917-be9b-f9ca42674988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:13.025390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:13.029454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1165ae-b29b-4917-be9b-f9ca42674988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:13.031193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1165ae-b29b-4917-be9b-f9ca42674988 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:13.053158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 659/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:29:17.994810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af985b-2026-4fb4-807c-f099bb0c5bd5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:17.995361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af985b-2026-4fb4-807c-f099bb0c5bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:17.995361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af985b-2026-4fb4-807c-f099bb0c5bd5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:17.996715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 691/1350] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:18.003350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf40d2e-2ee4-43e3-a3ef-02c874e286c5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:18.003790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf40d2e-2ee4-43e3-a3ef-02c874e286c5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:18.003888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf40d2e-2ee4-43e3-a3ef-02c874e286c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:18.179620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4bf40d2e-2ee4-43e3-a3ef-02c874e286c5 None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:18.180833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4bf40d2e-2ee4-43e3-a3ef-02c874e286c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:18.328865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 660/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:29:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:22.428417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068b9de-bc8f-488f-8ec6-75be46bd71e1 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:22.429086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068b9de-bc8f-488f-8ec6-75be46bd71e1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:22.429188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068b9de-bc8f-488f-8ec6-75be46bd71e1 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:22.430544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 692/1352] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:22.435741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855a53-76f1-42d8-92fc-a83f8c8a4933 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:22.436616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855a53-76f1-42d8-92fc-a83f8c8a4933 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:22.437126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855a53-76f1-42d8-92fc-a83f8c8a4933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:22.496331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:22.520903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c0855a53-76f1-42d8-92fc-a83f8c8a4933 None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:22.522795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c0855a53-76f1-42d8-92fc-a83f8c8a4933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:22.635894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 661/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:29:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:22.669038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=59 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:24.011712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6955c0-e441-4548-9c11-ac6624515b11 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:24.012363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6955c0-e441-4548-9c11-ac6624515b11 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:24.012363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6955c0-e441-4548-9c11-ac6624515b11 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:24.013647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 693/1354] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:24.018704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3d2c1-a5bd-48ed-bf21-f1b51f44b3d9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:24.019150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3d2c1-a5bd-48ed-bf21-f1b51f44b3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:24.019376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3d2c1-a5bd-48ed-bf21-f1b51f44b3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:24.098457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-47d3d2c1-a5bd-48ed-bf21-f1b51f44b3d9 None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:24.100279 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-47d3d2c1-a5bd-48ed-bf21-f1b51f44b3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:24.142079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:24.210146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 662/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:29:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:27.690834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849b7c-29cd-4362-ab14-daccc2294296 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:27.691374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849b7c-29cd-4362-ab14-daccc2294296 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:27.691374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849b7c-29cd-4362-ab14-daccc2294296 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:27.692746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 694/1356] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:29:27.703007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8b60e-4a8b-4340-95c9-4bc6d7fe5ac8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:27.704476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8b60e-4a8b-4340-95c9-4bc6d7fe5ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:27.704777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8b60e-4a8b-4340-95c9-4bc6d7fe5ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:27.799299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-81a8b60e-4a8b-4340-95c9-4bc6d7fe5ac8 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:27.801318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-81a8b60e-4a8b-4340-95c9-4bc6d7fe5ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:27.923767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 663/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:27.979816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d78c-19cd-4499-aaea-202230b568b4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:27.980597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d78c-19cd-4499-aaea-202230b568b4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:27.981211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d78c-19cd-4499-aaea-202230b568b4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:27.983489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 695/1358] 10.222.0.20 () {54 vars in 962 bytes} [Thu Jul 28 06:29:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:27.994016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507f09-ed40-4af9-a7f6-bc7ce2e432df None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:27.994610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507f09-ed40-4af9-a7f6-bc7ce2e432df None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:27.995097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507f09-ed40-4af9-a7f6-bc7ce2e432df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:28.016892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:28.080937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-47507f09-ed40-4af9-a7f6-bc7ce2e432df None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:28.082144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-47507f09-ed40-4af9-a7f6-bc7ce2e432df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:28.224852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 664/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Jul 28 06:29:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:28.234007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5d78c-19cd-4499-aaea-202230b568b4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:28.235751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aba5d78c-19cd-4499-aaea-202230b568b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:28.392174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5d78c-19cd-4499-aaea-202230b568b4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:28.475360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:28.523854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5f4b4-dd44-4122-9121-5f8517a4ccbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:28.524599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f4b4-dd44-4122-9121-5f8517a4ccbb service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:28.524972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f4b4-dd44-4122-9121-5f8517a4ccbb service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:28.525343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f4b4-dd44-4122-9121-5f8517a4ccbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:28.527186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9ec5f4b4-dd44-4122-9121-5f8517a4ccbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:28.624824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5f4b4-dd44-4122-9121-5f8517a4ccbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:28.626542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5f4b4-dd44-4122-9121-5f8517a4ccbb service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:28.646454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 696/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:29:30.922716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bf2f7-ea83-40b2-9bc5-00ac0ca5f1b9 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.923219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bf2f7-ea83-40b2-9bc5-00ac0ca5f1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.923219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bf2f7-ea83-40b2-9bc5-00ac0ca5f1b9 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:30.924697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 665/1361] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:30.930795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d753a2-fabc-4984-be51-68ceff29854d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:30.931193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d753a2-fabc-4984-be51-68ceff29854d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:30.931398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d753a2-fabc-4984-be51-68ceff29854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.003018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-66d753a2-fabc-4984-be51-68ceff29854d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:31.004263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-66d753a2-fabc-4984-be51-68ceff29854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:31.170527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:31.235610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 697/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:31.283289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bf2f7-ea83-40b2-9bc5-00ac0ca5f1b9 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:31.285028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ae9bf2f7-ea83-40b2-9bc5-00ac0ca5f1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:31.366934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:31.496427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bf2f7-ea83-40b2-9bc5-00ac0ca5f1b9 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:31.625395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23efcb-4140-4fd9-acb5-222b92cf6462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:31.626136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23efcb-4140-4fd9-acb5-222b92cf6462 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:31.626487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23efcb-4140-4fd9-acb5-222b92cf6462 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:31.626856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23efcb-4140-4fd9-acb5-222b92cf6462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:31.628560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ad23efcb-4140-4fd9-acb5-222b92cf6462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:31.654082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:31.727526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23efcb-4140-4fd9-acb5-222b92cf6462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:31.729257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23efcb-4140-4fd9-acb5-222b92cf6462 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:31.748516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 666/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:34.159400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61966922-06a1-4d1c-816b-152ebf46f472 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.159907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61966922-06a1-4d1c-816b-152ebf46f472 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.159907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61966922-06a1-4d1c-816b-152ebf46f472 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.165158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 698/1364] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:34.168002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c523c61-d8c4-4125-924e-74dfc3549aa1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.168452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c523c61-d8c4-4125-924e-74dfc3549aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.168792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c523c61-d8c4-4125-924e-74dfc3549aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.243985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1c523c61-d8c4-4125-924e-74dfc3549aa1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:34.245395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1c523c61-d8c4-4125-924e-74dfc3549aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:34.349859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:34.353064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 667/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:34.400718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65fb10-c3b0-4ac1-bf2c-7b201fec3327 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.401266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65fb10-c3b0-4ac1-bf2c-7b201fec3327 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.401643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65fb10-c3b0-4ac1-bf2c-7b201fec3327 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.403124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 699/1366] 10.222.0.20 () {54 vars in 962 bytes} [Thu Jul 28 06:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:34.409994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef45f9-b650-4d93-a4d7-aa29c403698e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.410350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef45f9-b650-4d93-a4d7-aa29c403698e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.410742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef45f9-b650-4d93-a4d7-aa29c403698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.497382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-caef45f9-b650-4d93-a4d7-aa29c403698e None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:34.498756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-caef45f9-b650-4d93-a4d7-aa29c403698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:34.629165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 668/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Jul 28 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:34.638529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65fb10-c3b0-4ac1-bf2c-7b201fec3327 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:34.640037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1c65fb10-c3b0-4ac1-bf2c-7b201fec3327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:34.795805 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65fb10-c3b0-4ac1-bf2c-7b201fec3327 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:34.817417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:34.918613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531969c7-e3ce-4ce3-b41c-3be1a2d899bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:34.919231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531969c7-e3ce-4ce3-b41c-3be1a2d899bd service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:34.919514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531969c7-e3ce-4ce3-b41c-3be1a2d899bd service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:34.919788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531969c7-e3ce-4ce3-b41c-3be1a2d899bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:34.921439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-531969c7-e3ce-4ce3-b41c-3be1a2d899bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:35.015350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531969c7-e3ce-4ce3-b41c-3be1a2d899bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:35.016934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531969c7-e3ce-4ce3-b41c-3be1a2d899bd service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:35.036437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 700/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:29:36.011315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b8cf-7da3-4997-871e-4d77c89a633b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.011879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b8cf-7da3-4997-871e-4d77c89a633b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.011879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b8cf-7da3-4997-871e-4d77c89a633b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.013410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 669/1369] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:36.018539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666c086-0f86-4968-956d-83607f01b526 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.018943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666c086-0f86-4968-956d-83607f01b526 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.019095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666c086-0f86-4968-956d-83607f01b526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.051172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:36.089551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1666c086-0f86-4968-956d-83607f01b526 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:36.090744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1666c086-0f86-4968-956d-83607f01b526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:36.191553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 701/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:36.538916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e1fb7-3ce2-4bfd-844e-2e1a8f314692 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.538916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e1fb7-3ce2-4bfd-844e-2e1a8f314692 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.538916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e1fb7-3ce2-4bfd-844e-2e1a8f314692 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.541608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 670/1371] 10.222.0.20 () {54 vars in 964 bytes} [Thu Jul 28 06:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:36.544990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd144cd-2635-4490-a6a3-6ee7a02ac937 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.545278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd144cd-2635-4490-a6a3-6ee7a02ac937 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.545594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd144cd-2635-4490-a6a3-6ee7a02ac937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.634727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7fd144cd-2635-4490-a6a3-6ee7a02ac937 None None] MFA Rules not processed for user `d6ee12d4109f4d8ba0d502558b9a2ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:36.635921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7fd144cd-2635-4490-a6a3-6ee7a02ac937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:36.685850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:36.718130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 702/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Thu Jul 28 06:29:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:36.725579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e1fb7-3ce2-4bfd-844e-2e1a8f314692 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:36.727074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-195e1fb7-3ce2-4bfd-844e-2e1a8f314692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:36.779664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:36.838362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e1fb7-3ce2-4bfd-844e-2e1a8f314692 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:36.916559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9656f5a-4af3-496a-84d6-30a4fc6f6ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:36.917181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656f5a-4af3-496a-84d6-30a4fc6f6ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:36.917505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656f5a-4af3-496a-84d6-30a4fc6f6ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:36.917764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656f5a-4af3-496a-84d6-30a4fc6f6ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:36.919396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b9656f5a-4af3-496a-84d6-30a4fc6f6ae5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:37.021376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:37.035027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9656f5a-4af3-496a-84d6-30a4fc6f6ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:37.036969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9656f5a-4af3-496a-84d6-30a4fc6f6ae5 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:37.056072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 671/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:29:37.165468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a39585-7f03-4593-ac51-6861bff9213c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:37.166017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a39585-7f03-4593-ac51-6861bff9213c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:37.166590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a39585-7f03-4593-ac51-6861bff9213c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:37.172136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 703/1374] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:37.173710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0f3ba-a96d-4ad0-84ad-ab85ebbaf524 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:37.174033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0f3ba-a96d-4ad0-84ad-ab85ebbaf524 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:37.174033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0f3ba-a96d-4ad0-84ad-ab85ebbaf524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:37.245436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-55d0f3ba-a96d-4ad0-84ad-ab85ebbaf524 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:37.246769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-55d0f3ba-a96d-4ad0-84ad-ab85ebbaf524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:37.352932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 672/1375] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:37.399743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a39585-7f03-4593-ac51-6861bff9213c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:37.401206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e3a39585-7f03-4593-ac51-6861bff9213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:37.590489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a39585-7f03-4593-ac51-6861bff9213c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:37.643089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:37.705440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d749ae6-9c6c-4800-85a1-f04980719f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:37.706057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d749ae6-9c6c-4800-85a1-f04980719f67 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:37.706336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d749ae6-9c6c-4800-85a1-f04980719f67 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:37.706601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d749ae6-9c6c-4800-85a1-f04980719f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:37.708279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9d749ae6-9c6c-4800-85a1-f04980719f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:37.803621 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d749ae6-9c6c-4800-85a1-f04980719f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:37.805211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d749ae6-9c6c-4800-85a1-f04980719f67 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:37.823885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 704/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:39.878281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815c615-99ab-43a2-bca6-0ce2ab25a737 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:39.878803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815c615-99ab-43a2-bca6-0ce2ab25a737 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:39.878902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815c615-99ab-43a2-bca6-0ce2ab25a737 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:39.880704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 673/1377] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:39.885943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf79cc0-5608-43b0-9bee-3834da1d06c4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:39.886371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf79cc0-5608-43b0-9bee-3834da1d06c4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:39.886490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf79cc0-5608-43b0-9bee-3834da1d06c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:39.952963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4cf79cc0-5608-43b0-9bee-3834da1d06c4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:39.954419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4cf79cc0-5608-43b0-9bee-3834da1d06c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:39.962755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:40.062312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 705/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:40.116078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815c615-99ab-43a2-bca6-0ce2ab25a737 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:40.117640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9815c615-99ab-43a2-bca6-0ce2ab25a737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:40.179958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:40.273012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815c615-99ab-43a2-bca6-0ce2ab25a737 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:40.393948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c0b4a-3757-4503-907e-cd0303cf1eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:40.394568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0b4a-3757-4503-907e-cd0303cf1eaf service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:40.394851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0b4a-3757-4503-907e-cd0303cf1eaf service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:40.395124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0b4a-3757-4503-907e-cd0303cf1eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:40.396875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ec8c0b4a-3757-4503-907e-cd0303cf1eaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:40.415118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:40.503381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c0b4a-3757-4503-907e-cd0303cf1eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:40.505727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c0b4a-3757-4503-907e-cd0303cf1eaf service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:40.537234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 674/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:42.643886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96db8ff-fc3d-4c39-a680-1f5824a4e4c4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:42.644425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96db8ff-fc3d-4c39-a680-1f5824a4e4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:42.644425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96db8ff-fc3d-4c39-a680-1f5824a4e4c4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:42.646804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 706/1380] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:42.652672 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d855c-7d7e-4194-b955-b13023e1733b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:42.653232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d855c-7d7e-4194-b955-b13023e1733b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:42.653232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d855c-7d7e-4194-b955-b13023e1733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:42.724375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cc8d855c-7d7e-4194-b955-b13023e1733b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:42.725910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cc8d855c-7d7e-4194-b955-b13023e1733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:42.823716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:42.833544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 675/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:42.876484 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96db8ff-fc3d-4c39-a680-1f5824a4e4c4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:42.878486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b96db8ff-fc3d-4c39-a680-1f5824a4e4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:43.023330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:43.034650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96db8ff-fc3d-4c39-a680-1f5824a4e4c4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:43.155919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f7ea-d1ba-45ff-a559-b799e4f3c37e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:43.156537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f7ea-d1ba-45ff-a559-b799e4f3c37e service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:43.156943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f7ea-d1ba-45ff-a559-b799e4f3c37e service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:43.157229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f7ea-d1ba-45ff-a559-b799e4f3c37e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:43.158883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8819f7ea-d1ba-45ff-a559-b799e4f3c37e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:43.259300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f7ea-d1ba-45ff-a559-b799e4f3c37e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:43.260902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819f7ea-d1ba-45ff-a559-b799e4f3c37e service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:43.272563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:43.282313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 707/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:45.516652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81b93-b8ee-4efc-8b8a-1f191962a19a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:45.517197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81b93-b8ee-4efc-8b8a-1f191962a19a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:45.517197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81b93-b8ee-4efc-8b8a-1f191962a19a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:45.518604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 676/1383] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:45.524281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a6a7-3b66-4f40-88ae-616297a8567e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:45.524722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a6a7-3b66-4f40-88ae-616297a8567e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:45.525141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a6a7-3b66-4f40-88ae-616297a8567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:45.622642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6192a6a7-3b66-4f40-88ae-616297a8567e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:45.624694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6192a6a7-3b66-4f40-88ae-616297a8567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:45.740597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 708/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:45.784889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e81b93-b8ee-4efc-8b8a-1f191962a19a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:45.787191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-45e81b93-b8ee-4efc-8b8a-1f191962a19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:45.978111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e81b93-b8ee-4efc-8b8a-1f191962a19a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:46.058759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:46.110225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c82e1-1382-4bf8-b9c1-0c8b6342cc59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:46.110875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c82e1-1382-4bf8-b9c1-0c8b6342cc59 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:46.111193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c82e1-1382-4bf8-b9c1-0c8b6342cc59 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:46.111518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c82e1-1382-4bf8-b9c1-0c8b6342cc59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:46.113210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f10c82e1-1382-4bf8-b9c1-0c8b6342cc59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:46.211806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c82e1-1382-4bf8-b9c1-0c8b6342cc59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:46.213513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c82e1-1382-4bf8-b9c1-0c8b6342cc59 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:46.235706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 677/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:48.389949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf473ee-a72a-4780-bc5c-593479ac47e4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:48.390620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf473ee-a72a-4780-bc5c-593479ac47e4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:48.390620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf473ee-a72a-4780-bc5c-593479ac47e4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:48.391829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 709/1386] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:48.398180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3c3a-f575-44ab-a781-1bd314c7e0e5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:48.398657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3c3a-f575-44ab-a781-1bd314c7e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:48.398991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a3c3a-f575-44ab-a781-1bd314c7e0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:48.474777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c13a3c3a-f575-44ab-a781-1bd314c7e0e5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:48.475996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c13a3c3a-f575-44ab-a781-1bd314c7e0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:48.514928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:48.588235 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 678/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:48.646883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf473ee-a72a-4780-bc5c-593479ac47e4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:48.648757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ebf473ee-a72a-4780-bc5c-593479ac47e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:48.740890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:48.826799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf473ee-a72a-4780-bc5c-593479ac47e4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:48.965593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4756425-0838-4daf-85c2-60ff8252a000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:48.966292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4756425-0838-4daf-85c2-60ff8252a000 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:48.966646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4756425-0838-4daf-85c2-60ff8252a000 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:48.966995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4756425-0838-4daf-85c2-60ff8252a000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:48.969035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d4756425-0838-4daf-85c2-60ff8252a000 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:49.016931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:49.078451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4756425-0838-4daf-85c2-60ff8252a000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:49.080051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4756425-0838-4daf-85c2-60ff8252a000 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:49.100893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 710/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:51.276482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bea38e-c4fc-4fe1-b053-2c75ab065e78 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:51.277184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bea38e-c4fc-4fe1-b053-2c75ab065e78 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:51.277184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bea38e-c4fc-4fe1-b053-2c75ab065e78 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:51.278483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 679/1389] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:51.284527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec24c4f-fc9e-496d-9b57-16085afacb4d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:51.285084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec24c4f-fc9e-496d-9b57-16085afacb4d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:51.285084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec24c4f-fc9e-496d-9b57-16085afacb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:51.351245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3ec24c4f-fc9e-496d-9b57-16085afacb4d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:51.352454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3ec24c4f-fc9e-496d-9b57-16085afacb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:51.453365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 711/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:51.495694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bea38e-c4fc-4fe1-b053-2c75ab065e78 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:51.497207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-67bea38e-c4fc-4fe1-b053-2c75ab065e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:51.653906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bea38e-c4fc-4fe1-b053-2c75ab065e78 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:51.672078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:51.773501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fee55-9148-464d-9158-2ca7f6c21c76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:51.774126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fee55-9148-464d-9158-2ca7f6c21c76 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:51.774413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fee55-9148-464d-9158-2ca7f6c21c76 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:51.774691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fee55-9148-464d-9158-2ca7f6c21c76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:51.776327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-851fee55-9148-464d-9158-2ca7f6c21c76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:51.875686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fee55-9148-464d-9158-2ca7f6c21c76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:51.877335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fee55-9148-464d-9158-2ca7f6c21c76 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:51.896563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 680/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:52.185203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1400d-d98b-406b-b0fd-ada21c2d6309 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:52.185440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1400d-d98b-406b-b0fd-ada21c2d6309 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:52.185720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1400d-d98b-406b-b0fd-ada21c2d6309 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:52.187216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 712/1392] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:52.192524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe2ad-b536-4551-857f-30a1c919b328 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:52.192816 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe2ad-b536-4551-857f-30a1c919b328 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:52.193095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe2ad-b536-4551-857f-30a1c919b328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:52.220402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:52.263273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e72fe2ad-b536-4551-857f-30a1c919b328 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:52.264483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e72fe2ad-b536-4551-857f-30a1c919b328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:52.368799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 681/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:52.426998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d6c10-bab8-4e27-b7be-e675114151b9 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:52.427477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d6c10-bab8-4e27-b7be-e675114151b9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:52.427684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d6c10-bab8-4e27-b7be-e675114151b9 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:52.429137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 713/1394] 10.222.0.20 () {54 vars in 966 bytes} [Thu Jul 28 06:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:52.434513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f13c0c-c2bc-4f2c-b922-71d6a01a55ec None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:52.434803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f13c0c-c2bc-4f2c-b922-71d6a01a55ec None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:52.435298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f13c0c-c2bc-4f2c-b922-71d6a01a55ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:52.512033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b5f13c0c-c2bc-4f2c-b922-71d6a01a55ec None None] MFA Rules not processed for user `649334bb7ed04bd69eb971fc070d96e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:52.513242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b5f13c0c-c2bc-4f2c-b922-71d6a01a55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:52.555179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:52.593078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 682/1395] 10.222.0.20 () {58 vars in 1063 bytes} [Thu Jul 28 06:29:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:52.601542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d6c10-bab8-4e27-b7be-e675114151b9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:52.603069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-720d6c10-bab8-4e27-b7be-e675114151b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:52.616885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:52.718674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d6c10-bab8-4e27-b7be-e675114151b9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:52.793610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b29f15-81d0-4447-ae8d-7bd73e086bc1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '649334bb7ed04bd69eb971fc070d96e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:52.794254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b29f15-81d0-4447-ae8d-7bd73e086bc1 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:52.794494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b29f15-81d0-4447-ae8d-7bd73e086bc1 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:52.794891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b29f15-81d0-4447-ae8d-7bd73e086bc1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:52.796508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-48b29f15-81d0-4447-ae8d-7bd73e086bc1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:52.866561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:52.907091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b29f15-81d0-4447-ae8d-7bd73e086bc1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:52.908624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b29f15-81d0-4447-ae8d-7bd73e086bc1 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:52.929471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 714/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:53.684017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab83e-2a54-4009-8aee-af99f9b4996b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:53.684504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab83e-2a54-4009-8aee-af99f9b4996b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:53.684615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab83e-2a54-4009-8aee-af99f9b4996b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:53.876059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-91bab83e-2a54-4009-8aee-af99f9b4996b None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:53.877189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-91bab83e-2a54-4009-8aee-af99f9b4996b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:53.964947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 683/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:29:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:53.993721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed0c6-9055-4ba7-8192-48f4695f3a23 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:53.994207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed0c6-9055-4ba7-8192-48f4695f3a23 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:53.994312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed0c6-9055-4ba7-8192-48f4695f3a23 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:53.995628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 715/1398] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:54.001149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914433-1036-41c8-b718-b58ce8cdac73 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.001633 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914433-1036-41c8-b718-b58ce8cdac73 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.001829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914433-1036-41c8-b718-b58ce8cdac73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.012254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f80f1f-886d-4fab-afc9-e2ebc3e3cb93 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.012628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f80f1f-886d-4fab-afc9-e2ebc3e3cb93 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.012961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f80f1f-886d-4fab-afc9-e2ebc3e3cb93 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.014606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 716/1399] 10.222.0.20 () {54 vars in 943 bytes} [Thu Jul 28 06:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:29:54.020721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3d45-a3fb-4781-860c-c582fb1d0c99 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.021098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3d45-a3fb-4781-860c-c582fb1d0c99 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.021470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3d45-a3fb-4781-860c-c582fb1d0c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.073238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4f914433-1036-41c8-b718-b58ce8cdac73 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:54.074506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4f914433-1036-41c8-b718-b58ce8cdac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:54.081114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:54.098491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-276a3d45-a3fb-4781-860c-c582fb1d0c99 None None] MFA Rules not processed for user `25dd5ceedc3040fdb75d9d24a78956a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:54.099711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-276a3d45-a3fb-4781-860c-c582fb1d0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:54.175170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 684/1400] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:54.210894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 717/1401] 10.222.0.20 () {58 vars in 1040 bytes} [Thu Jul 28 06:29:54 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:29:54.221666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f914433-1036-41c8-b718-b58ce8cdac73 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:54.222426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3d45-a3fb-4781-860c-c582fb1d0c99 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:54.222859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4f914433-1036-41c8-b718-b58ce8cdac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:54.227300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:54.233827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3d45-a3fb-4781-860c-c582fb1d0c99 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.340168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bc628-62b4-44ff-95a1-cbacc9a693db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.340791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc628-62b4-44ff-95a1-cbacc9a693db service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.341076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc628-62b4-44ff-95a1-cbacc9a693db service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.341385 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bc628-62b4-44ff-95a1-cbacc9a693db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.342414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f914433-1036-41c8-b718-b58ce8cdac73 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:54.343065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1d4bc628-62b4-44ff-95a1-cbacc9a693db service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:54.367631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:54.426531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bc628-62b4-44ff-95a1-cbacc9a693db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.428268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bc628-62b4-44ff-95a1-cbacc9a693db service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:54.438655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 718/1402] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:29:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 06:29:54.460527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be901928-965e-4948-bf0a-a9431aebb4c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:54.461255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be901928-965e-4948-bf0a-a9431aebb4c9 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:54.461629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be901928-965e-4948-bf0a-a9431aebb4c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:54.461969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be901928-965e-4948-bf0a-a9431aebb4c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:54.463655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-be901928-965e-4948-bf0a-a9431aebb4c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:54.580401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be901928-965e-4948-bf0a-a9431aebb4c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:54.582040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be901928-965e-4948-bf0a-a9431aebb4c9 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:54.601769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 685/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:56.711870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d98e3-febf-414d-b6c5-87d148a6a334 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:56.712505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d98e3-febf-414d-b6c5-87d148a6a334 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:56.712505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d98e3-febf-414d-b6c5-87d148a6a334 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:56.713777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 719/1404] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:56.719076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435f64c-b728-4217-908c-b46532b6c541 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:56.719544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435f64c-b728-4217-908c-b46532b6c541 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:56.719666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435f64c-b728-4217-908c-b46532b6c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:56.762494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:56.799278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1435f64c-b728-4217-908c-b46532b6c541 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:56.800503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1435f64c-b728-4217-908c-b46532b6c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:56.914463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 686/1405] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:56.962387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d98e3-febf-414d-b6c5-87d148a6a334 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:56.963847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-200d98e3-febf-414d-b6c5-87d148a6a334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:57.002998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:57.132049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d98e3-febf-414d-b6c5-87d148a6a334 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:29:57.257267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:57.268896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347701f-f6b2-46ed-8ff2-b2a3d3b7694c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:29:57.269544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347701f-f6b2-46ed-8ff2-b2a3d3b7694c service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:57.269813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347701f-f6b2-46ed-8ff2-b2a3d3b7694c service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:57.270695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347701f-f6b2-46ed-8ff2-b2a3d3b7694c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:57.272376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2347701f-f6b2-46ed-8ff2-b2a3d3b7694c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:57.373480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347701f-f6b2-46ed-8ff2-b2a3d3b7694c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:29:57.375064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347701f-f6b2-46ed-8ff2-b2a3d3b7694c service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:29:57.394506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 720/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:29:59.472168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520b81-98e2-458a-ab4d-349fef9744bd None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:59.474904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520b81-98e2-458a-ab4d-349fef9744bd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:59.475204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520b81-98e2-458a-ab4d-349fef9744bd None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:59.477177 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 687/1407] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:29:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:29:59.482612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe757d-201e-4a0d-83bd-6c26c932f224 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:29:59.483140 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe757d-201e-4a0d-83bd-6c26c932f224 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:29:59.483140 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe757d-201e-4a0d-83bd-6c26c932f224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:29:59.552120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-97fe757d-201e-4a0d-83bd-6c26c932f224 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:29:59.554957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-97fe757d-201e-4a0d-83bd-6c26c932f224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:59.637993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:59.676994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 721/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:29:59.724118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4520b81-98e2-458a-ab4d-349fef9744bd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:29:59.726038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a4520b81-98e2-458a-ab4d-349fef9744bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:29:59.899246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:29:59.955100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4520b81-98e2-458a-ab4d-349fef9744bd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:00.117550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3da7e9-6b18-470e-8c9e-f60035a728e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:00.118909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3da7e9-6b18-470e-8c9e-f60035a728e7 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:00.119361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3da7e9-6b18-470e-8c9e-f60035a728e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:00.119820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3da7e9-6b18-470e-8c9e-f60035a728e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:00.122640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0e3da7e9-6b18-470e-8c9e-f60035a728e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:00.220420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:00.245591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3da7e9-6b18-470e-8c9e-f60035a728e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:00.247197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3da7e9-6b18-470e-8c9e-f60035a728e7 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:00.267589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 688/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:09.674457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=14 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:10.263894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=14 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:16.994863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a6b09-1833-43d9-b025-464145b332b5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:16.995423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a6b09-1833-43d9-b025-464145b332b5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:16.995423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a6b09-1833-43d9-b025-464145b332b5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:16.996797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 722/1410] 10.222.0.20 () {54 vars in 918 bytes} [Thu Jul 28 06:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:17.001984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19f27-cc92-414b-8b39-3e6ff8f584cc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:17.002451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19f27-cc92-414b-8b39-3e6ff8f584cc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:17.002558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19f27-cc92-414b-8b39-3e6ff8f584cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:17.080769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cae19f27-cc92-414b-8b39-3e6ff8f584cc None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:17.083040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cae19f27-cc92-414b-8b39-3e6ff8f584cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:17.189323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 689/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Jul 28 06:30:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:17.204021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfc2d1-ba58-447d-b739-d31ed1fbedb5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:17.204309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfc2d1-ba58-447d-b739-d31ed1fbedb5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:17.204588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfc2d1-ba58-447d-b739-d31ed1fbedb5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:17.206040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 723/1412] 10.222.0.20 () {54 vars in 918 bytes} [Thu Jul 28 06:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:17.211238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a90fd-3022-4c00-a7ab-a334051ddc3b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:17.211533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a90fd-3022-4c00-a7ab-a334051ddc3b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:17.211817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a90fd-3022-4c00-a7ab-a334051ddc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:17.290462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:17.293147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a56a90fd-3022-4c00-a7ab-a334051ddc3b None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:17.295008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a56a90fd-3022-4c00-a7ab-a334051ddc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:17.409006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 690/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Jul 28 06:30:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:19.910471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77157467-46b7-4831-9dc9-98f7cf179f19 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.911046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77157467-46b7-4831-9dc9-98f7cf179f19 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.911046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77157467-46b7-4831-9dc9-98f7cf179f19 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.912521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 724/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:19.917756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f266d-bc1c-4643-9fde-5fcf581c02ee None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:19.918180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f266d-bc1c-4643-9fde-5fcf581c02ee None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:19.918327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f266d-bc1c-4643-9fde-5fcf581c02ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:19.992210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-813f266d-bc1c-4643-9fde-5fcf581c02ee None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:19.993459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-813f266d-bc1c-4643-9fde-5fcf581c02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:20.048181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:20.090726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 691/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:20.139716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77157467-46b7-4831-9dc9-98f7cf179f19 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:20.141398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-77157467-46b7-4831-9dc9-98f7cf179f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:20.293376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77157467-46b7-4831-9dc9-98f7cf179f19 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:20.365214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:20.405724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189695d-7e88-4ae5-94cf-4de858b178f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:20.406335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189695d-7e88-4ae5-94cf-4de858b178f8 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:20.406612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189695d-7e88-4ae5-94cf-4de858b178f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:20.406881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189695d-7e88-4ae5-94cf-4de858b178f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:20.408476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6189695d-7e88-4ae5-94cf-4de858b178f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:20.500746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189695d-7e88-4ae5-94cf-4de858b178f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:20.502371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189695d-7e88-4ae5-94cf-4de858b178f8 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:20.521267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 725/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:21.434727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefa2ee-a413-4cde-a9db-3d8f5b08999f None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:21.435317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefa2ee-a413-4cde-a9db-3d8f5b08999f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:21.435317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefa2ee-a413-4cde-a9db-3d8f5b08999f None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:21.436779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 692/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:21.442326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aa729-86c6-4a77-bce1-62dbb290f79b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:21.442779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aa729-86c6-4a77-bce1-62dbb290f79b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:21.442943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aa729-86c6-4a77-bce1-62dbb290f79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:21.519965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3d7aa729-86c6-4a77-bce1-62dbb290f79b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:21.521193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3d7aa729-86c6-4a77-bce1-62dbb290f79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:21.547937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:21.620487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 726/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:21.772416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefa2ee-a413-4cde-a9db-3d8f5b08999f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:21.773931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fcefa2ee-a413-4cde-a9db-3d8f5b08999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:21.924497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefa2ee-a413-4cde-a9db-3d8f5b08999f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:21.966947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:22.041780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffeaae-9832-4406-862f-def4caa43783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:22.042334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffeaae-9832-4406-862f-def4caa43783 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:22.042616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffeaae-9832-4406-862f-def4caa43783 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:22.042894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffeaae-9832-4406-862f-def4caa43783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:22.044516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-12ffeaae-9832-4406-862f-def4caa43783 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:22.140810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffeaae-9832-4406-862f-def4caa43783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:22.142810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffeaae-9832-4406-862f-def4caa43783 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:22.161848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 693/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:23.077837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1ded0-b642-46ea-b191-56e38848761d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:23.078402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1ded0-b642-46ea-b191-56e38848761d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:23.078402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1ded0-b642-46ea-b191-56e38848761d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:23.079720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 727/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:23.085612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ea182-a8b8-449f-beaf-9709d0d32bdf None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:23.086047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ea182-a8b8-449f-beaf-9709d0d32bdf None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:23.086175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ea182-a8b8-449f-beaf-9709d0d32bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:23.140422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:23.168555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-805ea182-a8b8-449f-beaf-9709d0d32bdf None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:23.169805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-805ea182-a8b8-449f-beaf-9709d0d32bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:23.266788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 694/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:23.422653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1ded0-b642-46ea-b191-56e38848761d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:23.424098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0cb1ded0-b642-46ea-b191-56e38848761d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:23.574168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1ded0-b642-46ea-b191-56e38848761d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:23.587009 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:23.689044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05fdd9-e00f-4080-8bea-99a98794bda0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:23.689616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05fdd9-e00f-4080-8bea-99a98794bda0 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:23.689898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05fdd9-e00f-4080-8bea-99a98794bda0 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:23.690168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05fdd9-e00f-4080-8bea-99a98794bda0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:23.692493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0d05fdd9-e00f-4080-8bea-99a98794bda0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:23.781740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05fdd9-e00f-4080-8bea-99a98794bda0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:23.783327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05fdd9-e00f-4080-8bea-99a98794bda0 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:23.801671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 728/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:24.666567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd88b1-9469-4f7d-9971-fc606f2b3fea None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:24.667095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd88b1-9469-4f7d-9971-fc606f2b3fea None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:24.667216 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd88b1-9469-4f7d-9971-fc606f2b3fea None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:24.668480 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 695/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:24.673476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19772-9072-414f-9ae4-a6227998098c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:24.673905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19772-9072-414f-9ae4-a6227998098c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:24.674056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19772-9072-414f-9ae4-a6227998098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:24.694072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:24.750914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-30e19772-9072-414f-9ae4-a6227998098c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:24.752153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-30e19772-9072-414f-9ae4-a6227998098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:24.849113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 729/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:30:25.001688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd88b1-9469-4f7d-9971-fc606f2b3fea None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:25.003097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a2fd88b1-9469-4f7d-9971-fc606f2b3fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:25.112759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:25.149854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd88b1-9469-4f7d-9971-fc606f2b3fea None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:25.263448 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c06e9-a7eb-4769-ae3f-4efac25de234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:25.263985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c06e9-a7eb-4769-ae3f-4efac25de234 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:25.264261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c06e9-a7eb-4769-ae3f-4efac25de234 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:25.264530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c06e9-a7eb-4769-ae3f-4efac25de234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:25.266169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-405c06e9-a7eb-4769-ae3f-4efac25de234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:25.340456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:25.356361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c06e9-a7eb-4769-ae3f-4efac25de234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:25.357984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c06e9-a7eb-4769-ae3f-4efac25de234 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:25.376021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 696/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:26.239473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0ef9-0769-497a-9a8b-b75047ff6e80 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:26.240034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0ef9-0769-497a-9a8b-b75047ff6e80 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:26.240137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0ef9-0769-497a-9a8b-b75047ff6e80 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:26.241763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 730/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:26.246717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e58ab-3469-465b-adbf-d3e37218ab06 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:26.247129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e58ab-3469-465b-adbf-d3e37218ab06 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:26.247277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e58ab-3469-465b-adbf-d3e37218ab06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:26.324507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4d4e58ab-3469-465b-adbf-d3e37218ab06 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:26.325742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4d4e58ab-3469-465b-adbf-d3e37218ab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:26.420927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 697/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:26.573820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0ef9-0769-497a-9a8b-b75047ff6e80 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:26.575268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-57be0ef9-0769-497a-9a8b-b75047ff6e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:26.660025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:26.728760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0ef9-0769-497a-9a8b-b75047ff6e80 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:26.842707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778b927-eac3-43c3-86d3-08a732a15faa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:26.843258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778b927-eac3-43c3-86d3-08a732a15faa service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:26.843539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778b927-eac3-43c3-86d3-08a732a15faa service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:26.843812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778b927-eac3-43c3-86d3-08a732a15faa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:26.845492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6778b927-eac3-43c3-86d3-08a732a15faa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:26.894311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:26.935457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778b927-eac3-43c3-86d3-08a732a15faa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:26.937056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778b927-eac3-43c3-86d3-08a732a15faa service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:26.955681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 731/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:27.791869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d426e9-24a8-4eb7-895c-4c20cd369560 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:27.792795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d426e9-24a8-4eb7-895c-4c20cd369560 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:27.792795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d426e9-24a8-4eb7-895c-4c20cd369560 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:27.793842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 698/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:27.799398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb129ab-ae54-45c0-955e-986d34c862bc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:27.799920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb129ab-ae54-45c0-955e-986d34c862bc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:27.800025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb129ab-ae54-45c0-955e-986d34c862bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:27.883332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8eb129ab-ae54-45c0-955e-986d34c862bc None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:27.884589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8eb129ab-ae54-45c0-955e-986d34c862bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:28.015985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 732/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:28.186641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d426e9-24a8-4eb7-895c-4c20cd369560 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:28.188959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-63d426e9-24a8-4eb7-895c-4c20cd369560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:28.245260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:28.348547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d426e9-24a8-4eb7-895c-4c20cd369560 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:28.474078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8743118-9691-421a-bce9-37ef89e2bb4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:28.474706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743118-9691-421a-bce9-37ef89e2bb4d service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:28.475272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743118-9691-421a-bce9-37ef89e2bb4d service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:28.475555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743118-9691-421a-bce9-37ef89e2bb4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:28.477171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e8743118-9691-421a-bce9-37ef89e2bb4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:28.484055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:28.568423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8743118-9691-421a-bce9-37ef89e2bb4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:28.570075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8743118-9691-421a-bce9-37ef89e2bb4d service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:28.588802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 699/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:30:29.405388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd2e78-a409-48fc-b96a-cf15fe083dd1 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:29.405940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd2e78-a409-48fc-b96a-cf15fe083dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:29.405940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd2e78-a409-48fc-b96a-cf15fe083dd1 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:29.407306 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 733/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:29.412899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80db8c3-fcd9-4931-84ee-e2bfd6e7da1b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:29.413388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80db8c3-fcd9-4931-84ee-e2bfd6e7da1b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:29.413500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80db8c3-fcd9-4931-84ee-e2bfd6e7da1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:29.492143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f80db8c3-fcd9-4931-84ee-e2bfd6e7da1b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:29.493396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f80db8c3-fcd9-4931-84ee-e2bfd6e7da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:29.560236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:29.592150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 700/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:29.745774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd2e78-a409-48fc-b96a-cf15fe083dd1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:29.747304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c1fd2e78-a409-48fc-b96a-cf15fe083dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:29.765335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:29.896438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd2e78-a409-48fc-b96a-cf15fe083dd1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:29.994460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:30.027520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78217f-3f28-4df3-b72d-4a06a1396d0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:30.028054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78217f-3f28-4df3-b72d-4a06a1396d0d service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:30.028337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78217f-3f28-4df3-b72d-4a06a1396d0d service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:30.028611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78217f-3f28-4df3-b72d-4a06a1396d0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:30.030302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8d78217f-3f28-4df3-b72d-4a06a1396d0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:30.123745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78217f-3f28-4df3-b72d-4a06a1396d0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:30.125396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78217f-3f28-4df3-b72d-4a06a1396d0d service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:30.145020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 734/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:30.966767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1e8f-9be5-40b1-8535-9f215e98ca83 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:30.967357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1e8f-9be5-40b1-8535-9f215e98ca83 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:30.967357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1e8f-9be5-40b1-8535-9f215e98ca83 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:30.968707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 701/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:30.974424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3358a9-f099-4927-805a-d63d3bb573a4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:30.974860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3358a9-f099-4927-805a-d63d3bb573a4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:30.974990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3358a9-f099-4927-805a-d63d3bb573a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:31.051593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1f3358a9-f099-4927-805a-d63d3bb573a4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:31.052803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1f3358a9-f099-4927-805a-d63d3bb573a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:31.094207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:31.150106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 735/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:31.302751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e1e8f-9be5-40b1-8535-9f215e98ca83 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:31.304296 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-228e1e8f-9be5-40b1-8535-9f215e98ca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:31.452790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e1e8f-9be5-40b1-8535-9f215e98ca83 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:31.507780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:31.567537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6f859-1f09-4a2d-be72-4673374e2228 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:31.568071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f859-1f09-4a2d-be72-4673374e2228 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:31.568350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f859-1f09-4a2d-be72-4673374e2228 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:31.568615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6f859-1f09-4a2d-be72-4673374e2228 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:31.570502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ffc6f859-1f09-4a2d-be72-4673374e2228 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:31.660738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6f859-1f09-4a2d-be72-4673374e2228 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:31.662358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6f859-1f09-4a2d-be72-4673374e2228 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:31.680909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 702/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:32.488296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184ffec-2d0c-42e7-bd64-ef300404d81c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.489408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184ffec-2d0c-42e7-bd64-ef300404d81c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.489408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184ffec-2d0c-42e7-bd64-ef300404d81c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.490223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 736/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:32.495193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b04e4-c4dc-46c7-939b-d1dd8d1fc0c5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:32.495649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b04e4-c4dc-46c7-939b-d1dd8d1fc0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:32.495751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b04e4-c4dc-46c7-939b-d1dd8d1fc0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:32.573757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-604b04e4-c4dc-46c7-939b-d1dd8d1fc0c5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:32.574959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-604b04e4-c4dc-46c7-939b-d1dd8d1fc0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:32.576970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:32.675274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 703/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:32.828020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184ffec-2d0c-42e7-bd64-ef300404d81c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:32.829603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b184ffec-2d0c-42e7-bd64-ef300404d81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:32.975759 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184ffec-2d0c-42e7-bd64-ef300404d81c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:33.002698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:33.089338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132b127-74ac-4050-91fe-5724b9a6fea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:33.089959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b127-74ac-4050-91fe-5724b9a6fea3 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:33.090242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b127-74ac-4050-91fe-5724b9a6fea3 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:33.090519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b127-74ac-4050-91fe-5724b9a6fea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:33.092144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8132b127-74ac-4050-91fe-5724b9a6fea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:33.182661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132b127-74ac-4050-91fe-5724b9a6fea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:33.184266 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132b127-74ac-4050-91fe-5724b9a6fea3 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:33.202621 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 737/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:35.119995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc5c33-2e03-4441-bd1a-4b639ddd39d6 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:35.120562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc5c33-2e03-4441-bd1a-4b639ddd39d6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:35.120562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc5c33-2e03-4441-bd1a-4b639ddd39d6 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:35.121913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 704/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:35.127058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbeba8-30a7-4b5d-accc-ec20198cf256 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:35.127468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbeba8-30a7-4b5d-accc-ec20198cf256 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:35.127608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbeba8-30a7-4b5d-accc-ec20198cf256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:35.170135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:35.195824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c8dbeba8-30a7-4b5d-accc-ec20198cf256 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:35.197033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c8dbeba8-30a7-4b5d-accc-ec20198cf256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:35.297745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 738/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:35.336739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc5c33-2e03-4441-bd1a-4b639ddd39d6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:35.338299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-41fc5c33-2e03-4441-bd1a-4b639ddd39d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:35.447122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc5c33-2e03-4441-bd1a-4b639ddd39d6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:35.480934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:35.522319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6a683-ae4c-46c9-9ba2-79adaa109f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:35.522952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a683-ae4c-46c9-9ba2-79adaa109f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:35.523230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a683-ae4c-46c9-9ba2-79adaa109f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:35.523497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a683-ae4c-46c9-9ba2-79adaa109f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:35.525952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-12a6a683-ae4c-46c9-9ba2-79adaa109f6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:35.643987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6a683-ae4c-46c9-9ba2-79adaa109f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:35.645741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6a683-ae4c-46c9-9ba2-79adaa109f6a service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:35.665530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 705/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:37.822445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16416-c07a-4ef4-a215-8820fbca8037 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:37.822995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16416-c07a-4ef4-a215-8820fbca8037 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:37.822995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16416-c07a-4ef4-a215-8820fbca8037 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:37.824360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 739/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:37.829739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ae86-a0d3-49e0-8f94-b3eefe4a1d3c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:37.830179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ae86-a0d3-49e0-8f94-b3eefe4a1d3c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:37.830285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ae86-a0d3-49e0-8f94-b3eefe4a1d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:37.897189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3447ae86-a0d3-49e0-8f94-b3eefe4a1d3c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:37.898444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3447ae86-a0d3-49e0-8f94-b3eefe4a1d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:37.911797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:38.000758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 706/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:38.039980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a16416-c07a-4ef4-a215-8820fbca8037 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:38.041531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-83a16416-c07a-4ef4-a215-8820fbca8037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:38.157605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a16416-c07a-4ef4-a215-8820fbca8037 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:38.170728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:38.243256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dd4b-2ae0-42fc-bbb6-0fc1d2f5a8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:38.243882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dd4b-2ae0-42fc-bbb6-0fc1d2f5a8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:38.244177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dd4b-2ae0-42fc-bbb6-0fc1d2f5a8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:38.244521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dd4b-2ae0-42fc-bbb6-0fc1d2f5a8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:38.246178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1bf2dd4b-2ae0-42fc-bbb6-0fc1d2f5a8f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:38.365855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2dd4b-2ae0-42fc-bbb6-0fc1d2f5a8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:38.367365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2dd4b-2ae0-42fc-bbb6-0fc1d2f5a8f9 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:38.386401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 740/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:40.792157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8e76-c541-45bb-bd15-7761d91a8c5b None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:40.792157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8e76-c541-45bb-bd15-7761d91a8c5b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:40.792157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8e76-c541-45bb-bd15-7761d91a8c5b None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:40.792157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 707/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:40.797693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ad38d-7865-4de2-8ea8-755c0b64d1eb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:40.798109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ad38d-7865-4de2-8ea8-755c0b64d1eb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:40.798259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ad38d-7865-4de2-8ea8-755c0b64d1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:40.850381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:40.868960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8e0ad38d-7865-4de2-8ea8-755c0b64d1eb None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:40.870328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8e0ad38d-7865-4de2-8ea8-755c0b64d1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:40.970910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 741/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:41.014259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d8e76-c541-45bb-bd15-7761d91a8c5b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:41.015787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ad2d8e76-c541-45bb-bd15-7761d91a8c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:41.126861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d8e76-c541-45bb-bd15-7761d91a8c5b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:41.168612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:41.205334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534b12-fd87-4f03-80ad-f73f56556b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:41.205964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534b12-fd87-4f03-80ad-f73f56556b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:41.206369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534b12-fd87-4f03-80ad-f73f56556b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:41.207720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534b12-fd87-4f03-80ad-f73f56556b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:41.209445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a0534b12-fd87-4f03-80ad-f73f56556b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:41.327865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534b12-fd87-4f03-80ad-f73f56556b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:41.329322 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534b12-fd87-4f03-80ad-f73f56556b30 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:41.348463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 708/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:43.558378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949b525-7476-4f3d-96e9-690a28453d8b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.558934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949b525-7476-4f3d-96e9-690a28453d8b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.558934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949b525-7476-4f3d-96e9-690a28453d8b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.560280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 742/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:43.565558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f38f66-3dc8-4459-91de-296c6e280a8a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.566596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f38f66-3dc8-4459-91de-296c6e280a8a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.566699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f38f66-3dc8-4459-91de-296c6e280a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.637991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-87f38f66-3dc8-4459-91de-296c6e280a8a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:43.639495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-87f38f66-3dc8-4459-91de-296c6e280a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:43.660948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:43.741589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 709/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:43.782566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949b525-7476-4f3d-96e9-690a28453d8b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:43.784185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7949b525-7476-4f3d-96e9-690a28453d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:43.894661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949b525-7476-4f3d-96e9-690a28453d8b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:43.915735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:43.975252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d329b2-fbbd-4035-941e-dd0ba7db200c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:43.975871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d329b2-fbbd-4035-941e-dd0ba7db200c service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:43.976154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d329b2-fbbd-4035-941e-dd0ba7db200c service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:43.976428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d329b2-fbbd-4035-941e-dd0ba7db200c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:43.978115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-58d329b2-fbbd-4035-941e-dd0ba7db200c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:44.097776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d329b2-fbbd-4035-941e-dd0ba7db200c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:44.099511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d329b2-fbbd-4035-941e-dd0ba7db200c service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:44.120710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 743/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:30:46.252303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbe4e1-7c28-4ff5-8ba8-5cd5236b2fab None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:46.252824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbe4e1-7c28-4ff5-8ba8-5cd5236b2fab None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:46.253503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbe4e1-7c28-4ff5-8ba8-5cd5236b2fab None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:46.254837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 710/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:46.260207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4468-1711-439f-851c-9c7670e3ea20 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:46.260639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4468-1711-439f-851c-9c7670e3ea20 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:46.260787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4468-1711-439f-851c-9c7670e3ea20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:46.330568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:46.334792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d6fd4468-1711-439f-851c-9c7670e3ea20 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:46.336017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d6fd4468-1711-439f-851c-9c7670e3ea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:46.439900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 744/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:46.481956 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbe4e1-7c28-4ff5-8ba8-5cd5236b2fab None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:46.483481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8fbbe4e1-7c28-4ff5-8ba8-5cd5236b2fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:46.589865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbe4e1-7c28-4ff5-8ba8-5cd5236b2fab None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:46.640880 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:46.665720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d6209-d98f-4dbf-b255-2c35f83e825d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:46.666343 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d6209-d98f-4dbf-b255-2c35f83e825d service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:46.666627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d6209-d98f-4dbf-b255-2c35f83e825d service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:46.666899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d6209-d98f-4dbf-b255-2c35f83e825d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:46.668510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-717d6209-d98f-4dbf-b255-2c35f83e825d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:46.779748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d6209-d98f-4dbf-b255-2c35f83e825d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:46.781188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d6209-d98f-4dbf-b255-2c35f83e825d service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:46.799006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 711/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:48.980118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1acf41-1b86-4b46-8db0-de25d54c4eb4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:48.980697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1acf41-1b86-4b46-8db0-de25d54c4eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:48.980697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1acf41-1b86-4b46-8db0-de25d54c4eb4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:48.982075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 745/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:48.987982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919bb58-324c-4911-b2fa-0859da7b8016 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:48.987982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919bb58-324c-4911-b2fa-0859da7b8016 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:48.988918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919bb58-324c-4911-b2fa-0859da7b8016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:49.054014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a919bb58-324c-4911-b2fa-0859da7b8016 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:49.055216 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a919bb58-324c-4911-b2fa-0859da7b8016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:49.084212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:49.152315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 712/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:49.191574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1acf41-1b86-4b46-8db0-de25d54c4eb4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:49.193100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9a1acf41-1b86-4b46-8db0-de25d54c4eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:49.297640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1acf41-1b86-4b46-8db0-de25d54c4eb4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:49.326092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:49.370321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae23bb4-0664-47fe-b8b5-61c3634e04be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:49.370930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae23bb4-0664-47fe-b8b5-61c3634e04be service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:49.371209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae23bb4-0664-47fe-b8b5-61c3634e04be service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:49.371481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae23bb4-0664-47fe-b8b5-61c3634e04be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:49.373084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aae23bb4-0664-47fe-b8b5-61c3634e04be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:49.482189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae23bb4-0664-47fe-b8b5-61c3634e04be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:49.483625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae23bb4-0664-47fe-b8b5-61c3634e04be service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:49.503905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 746/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:51.551122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d81429-1141-47f3-9d6c-792bd230a1c4 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:51.551655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d81429-1141-47f3-9d6c-792bd230a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:51.551655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d81429-1141-47f3-9d6c-792bd230a1c4 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:51.553006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 713/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:51.558237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398056-809b-49c1-881d-a5d7a7ed3027 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:51.558643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398056-809b-49c1-881d-a5d7a7ed3027 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:51.558800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398056-809b-49c1-881d-a5d7a7ed3027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:51.625865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-83398056-809b-49c1-881d-a5d7a7ed3027 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:51.627070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-83398056-809b-49c1-881d-a5d7a7ed3027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:51.636007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:51.724686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 747/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:51.762227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d81429-1141-47f3-9d6c-792bd230a1c4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:51.763801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b0d81429-1141-47f3-9d6c-792bd230a1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:51.866983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d81429-1141-47f3-9d6c-792bd230a1c4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:51.922716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:51.941072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d23e0-49fb-4d6b-bbce-c90cde8e1cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:51.941714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d23e0-49fb-4d6b-bbce-c90cde8e1cef service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:51.941999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d23e0-49fb-4d6b-bbce-c90cde8e1cef service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:51.942273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d23e0-49fb-4d6b-bbce-c90cde8e1cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:51.943885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-824d23e0-49fb-4d6b-bbce-c90cde8e1cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:52.052181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d23e0-49fb-4d6b-bbce-c90cde8e1cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:52.053571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d23e0-49fb-4d6b-bbce-c90cde8e1cef service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:52.071488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 714/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:54.133528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f17f4c-4554-459c-b61b-bf26892adbec None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:54.134096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f17f4c-4554-459c-b61b-bf26892adbec None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:54.134096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f17f4c-4554-459c-b61b-bf26892adbec None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:54.135460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 748/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:54.140591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0b128-04d6-4fdc-8168-2a69143ebf1d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:54.141022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0b128-04d6-4fdc-8168-2a69143ebf1d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:54.141138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0b128-04d6-4fdc-8168-2a69143ebf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:54.210490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d9d0b128-04d6-4fdc-8168-2a69143ebf1d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:54.211697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d9d0b128-04d6-4fdc-8168-2a69143ebf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:54.250792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:54.309670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 715/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:54.349175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f17f4c-4554-459c-b61b-bf26892adbec None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:54.350743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-12f17f4c-4554-459c-b61b-bf26892adbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:54.451330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f17f4c-4554-459c-b61b-bf26892adbec None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:54.488960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:54.525316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1107b1-45f7-4220-bc7f-5882e966b101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:54.525937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1107b1-45f7-4220-bc7f-5882e966b101 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:54.526222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1107b1-45f7-4220-bc7f-5882e966b101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:54.526497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1107b1-45f7-4220-bc7f-5882e966b101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:54.528118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd1107b1-45f7-4220-bc7f-5882e966b101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:54.636342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1107b1-45f7-4220-bc7f-5882e966b101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:54.637816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1107b1-45f7-4220-bc7f-5882e966b101 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:54.656285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 749/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:56.717724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934d6d0-58ea-4776-bc58-391834396b40 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:56.718273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934d6d0-58ea-4776-bc58-391834396b40 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:56.718273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934d6d0-58ea-4776-bc58-391834396b40 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:56.719560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 716/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:56.725596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3333a2-c4e2-4266-a067-25d2ca80b07f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:56.726026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3333a2-c4e2-4266-a067-25d2ca80b07f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:56.726158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3333a2-c4e2-4266-a067-25d2ca80b07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:56.796246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1a3333a2-c4e2-4266-a067-25d2ca80b07f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:56.797514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1a3333a2-c4e2-4266-a067-25d2ca80b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:56.816158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:56.893850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 750/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:56.931191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934d6d0-58ea-4776-bc58-391834396b40 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:56.932746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7934d6d0-58ea-4776-bc58-391834396b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:57.035112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934d6d0-58ea-4776-bc58-391834396b40 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:57.101411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:57.108723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccf33f-cff6-462e-9b92-64bc54499a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:57.109371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccf33f-cff6-462e-9b92-64bc54499a3d service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:57.109660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccf33f-cff6-462e-9b92-64bc54499a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:57.109937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccf33f-cff6-462e-9b92-64bc54499a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:57.111546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-aaccf33f-cff6-462e-9b92-64bc54499a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:57.220444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccf33f-cff6-462e-9b92-64bc54499a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:57.222681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccf33f-cff6-462e-9b92-64bc54499a3d service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:57.240456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 717/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:30:59.319582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449a771-51a1-4591-a519-5506da2aaf77 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.320180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449a771-51a1-4591-a519-5506da2aaf77 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.320180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449a771-51a1-4591-a519-5506da2aaf77 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.321788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 751/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:30:59.327001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17612-6ba5-4b46-84dc-1fde9b1fa191 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.327451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17612-6ba5-4b46-84dc-1fde9b1fa191 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.327553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17612-6ba5-4b46-84dc-1fde9b1fa191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.392487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-78f17612-6ba5-4b46-84dc-1fde9b1fa191 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:30:59.393750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-78f17612-6ba5-4b46-84dc-1fde9b1fa191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:59.441993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:59.494480 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 718/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:30:59.531203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449a771-51a1-4591-a519-5506da2aaf77 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:30:59.532737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3449a771-51a1-4591-a519-5506da2aaf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:59.638249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449a771-51a1-4591-a519-5506da2aaf77 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:30:59.686399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:30:59.712997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8bb70-1774-4fe7-83e7-700a495e330d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:30:59.713627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8bb70-1774-4fe7-83e7-700a495e330d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:30:59.713912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8bb70-1774-4fe7-83e7-700a495e330d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:30:59.714185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8bb70-1774-4fe7-83e7-700a495e330d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:30:59.715811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c1f8bb70-1774-4fe7-83e7-700a495e330d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:30:59.823400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8bb70-1774-4fe7-83e7-700a495e330d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:30:59.824824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8bb70-1774-4fe7-83e7-700a495e330d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:30:59.842997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 752/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:09.492639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=22 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:09.840704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=66 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:19.558247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e46f2-2107-40f2-855e-5ede94608874 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:19.558782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e46f2-2107-40f2-855e-5ede94608874 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:19.558891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e46f2-2107-40f2-855e-5ede94608874 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:19.560192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 719/1471] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:19.566021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097448b8-dc59-4599-b688-f14bc284ebfd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:19.566471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097448b8-dc59-4599-b688-f14bc284ebfd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:19.566575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097448b8-dc59-4599-b688-f14bc284ebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:19.635972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-097448b8-dc59-4599-b688-f14bc284ebfd None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:19.637168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-097448b8-dc59-4599-b688-f14bc284ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:19.732022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 753/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:19.817197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72486d57-055f-45ba-b042-3308de252ac8 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:19.817585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72486d57-055f-45ba-b042-3308de252ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:19.817862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72486d57-055f-45ba-b042-3308de252ac8 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:19.819193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 720/1473] 10.222.0.20 () {54 vars in 966 bytes} [Thu Jul 28 06:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:19.824127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f08d4-6553-4c29-bccb-229ce9b9aaab None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:19.824434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f08d4-6553-4c29-bccb-229ce9b9aaab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:19.824706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f08d4-6553-4c29-bccb-229ce9b9aaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:19.898639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-be9f08d4-6553-4c29-bccb-229ce9b9aaab None None] MFA Rules not processed for user `649334bb7ed04bd69eb971fc070d96e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:19.899836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-be9f08d4-6553-4c29-bccb-229ce9b9aaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:19.920492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:19.977351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 754/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Thu Jul 28 06:31:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:19.984980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72486d57-055f-45ba-b042-3308de252ac8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:19.987263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-72486d57-055f-45ba-b042-3308de252ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:20.090699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72486d57-055f-45ba-b042-3308de252ac8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:20.162776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab70648-550e-4947-bbc6-90da686d5b15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '649334bb7ed04bd69eb971fc070d96e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:20.163392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70648-550e-4947-bbc6-90da686d5b15 service glance] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:20.163675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70648-550e-4947-bbc6-90da686d5b15 service glance] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:20.163952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70648-550e-4947-bbc6-90da686d5b15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:20.165605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5ab70648-550e-4947-bbc6-90da686d5b15 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:20.219453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:20.274331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab70648-550e-4947-bbc6-90da686d5b15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:20.275683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab70648-550e-4947-bbc6-90da686d5b15 service glance] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:20.295814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 721/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:22.421230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23790d-bc72-4d44-b53d-10e13d2df412 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:22.421796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23790d-bc72-4d44-b53d-10e13d2df412 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:22.421796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23790d-bc72-4d44-b53d-10e13d2df412 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:22.423168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 755/1476] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:22.428357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ce04-cef2-4a24-9d1e-e9b9157acc61 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:22.428762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ce04-cef2-4a24-9d1e-e9b9157acc61 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:22.428924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ce04-cef2-4a24-9d1e-e9b9157acc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:22.494790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c141ce04-cef2-4a24-9d1e-e9b9157acc61 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:22.496009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c141ce04-cef2-4a24-9d1e-e9b9157acc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:22.593695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 722/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:22.618505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23790d-bc72-4d44-b53d-10e13d2df412 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:22.620024 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-df23790d-bc72-4d44-b53d-10e13d2df412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:22.746915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23790d-bc72-4d44-b53d-10e13d2df412 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:22.794993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:22.840627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6203a34-7e48-4639-b694-43fb52abf4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:22.841253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6203a34-7e48-4639-b694-43fb52abf4ca admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:22.841568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6203a34-7e48-4639-b694-43fb52abf4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:22.842115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6203a34-7e48-4639-b694-43fb52abf4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:22.842823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6203a34-7e48-4639-b694-43fb52abf4ca admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:22.844538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-d6203a34-7e48-4639-b694-43fb52abf4ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:31:22.844538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 28 06:31:22.844538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: Jul 28 06:31:22.845717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6203a34-7e48-4639-b694-43fb52abf4ca admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:22.865625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 756/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Jul 28 06:31:22 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:31:24.877907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70ef12-d343-4c7a-b377-0d5cf029e29a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.878450 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70ef12-d343-4c7a-b377-0d5cf029e29a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.878450 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70ef12-d343-4c7a-b377-0d5cf029e29a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.879810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 723/1479] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:24.885133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a877e-7da0-4856-b039-fc1f039aec5c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:24.885601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a877e-7da0-4856-b039-fc1f039aec5c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:24.885710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a877e-7da0-4856-b039-fc1f039aec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:24.952682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-468a877e-7da0-4856-b039-fc1f039aec5c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:24.953896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-468a877e-7da0-4856-b039-fc1f039aec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:25.049111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 757/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:25.093371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70ef12-d343-4c7a-b377-0d5cf029e29a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:25.094996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7a70ef12-d343-4c7a-b377-0d5cf029e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:25.106129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:25.242943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70ef12-d343-4c7a-b377-0d5cf029e29a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:25.323596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:25.356190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6491da-8463-41d6-9a52-cdf289c02347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:25.356801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6491da-8463-41d6-9a52-cdf289c02347 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:25.357081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6491da-8463-41d6-9a52-cdf289c02347 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:25.357375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6491da-8463-41d6-9a52-cdf289c02347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:25.358974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2c6491da-8463-41d6-9a52-cdf289c02347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:25.447341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6491da-8463-41d6-9a52-cdf289c02347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:25.448957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6491da-8463-41d6-9a52-cdf289c02347 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:25.467556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 724/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:27.665754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364c53d-5a64-4618-a3c1-522cbf860b9b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:27.666288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364c53d-5a64-4618-a3c1-522cbf860b9b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:27.666426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364c53d-5a64-4618-a3c1-522cbf860b9b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:27.667706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 758/1482] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:27.673044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3969d-5b12-4402-9aec-a00ddceacc6a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:27.673513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3969d-5b12-4402-9aec-a00ddceacc6a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:27.673619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3969d-5b12-4402-9aec-a00ddceacc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:27.746221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-46b3969d-5b12-4402-9aec-a00ddceacc6a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:27.747891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-46b3969d-5b12-4402-9aec-a00ddceacc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:27.802293 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:27.855651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 725/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:27.894085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364c53d-5a64-4618-a3c1-522cbf860b9b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:27.896722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3364c53d-5a64-4618-a3c1-522cbf860b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:27.926690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:28.023682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364c53d-5a64-4618-a3c1-522cbf860b9b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:28.110474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2367b-8680-461d-ae48-97c373af0838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:28.111126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2367b-8680-461d-ae48-97c373af0838 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:28.111961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2367b-8680-461d-ae48-97c373af0838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:28.112298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2367b-8680-461d-ae48-97c373af0838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:28.114371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6aa2367b-8680-461d-ae48-97c373af0838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:28.207810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:28.253736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2367b-8680-461d-ae48-97c373af0838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:28.255637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2367b-8680-461d-ae48-97c373af0838 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:28.275092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 759/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:30.356419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13340a-efa7-4818-bc5e-4f7cb392c9d8 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:30.356958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13340a-efa7-4818-bc5e-4f7cb392c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:30.356958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13340a-efa7-4818-bc5e-4f7cb392c9d8 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:30.358475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 726/1485] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:30.364244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457b3d9-bab6-4aea-9ed1-67d24c7a869a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:30.364717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457b3d9-bab6-4aea-9ed1-67d24c7a869a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:30.364829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457b3d9-bab6-4aea-9ed1-67d24c7a869a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:30.431248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8457b3d9-bab6-4aea-9ed1-67d24c7a869a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:30.432458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8457b3d9-bab6-4aea-9ed1-67d24c7a869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:30.539378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 760/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:30.578854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13340a-efa7-4818-bc5e-4f7cb392c9d8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:30.580964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-da13340a-efa7-4818-bc5e-4f7cb392c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:30.684608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13340a-efa7-4818-bc5e-4f7cb392c9d8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:30.758149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:30.759998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72d40f-26c3-498d-a20c-f3d82291539f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:30.760633 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d40f-26c3-498d-a20c-f3d82291539f service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:30.761113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d40f-26c3-498d-a20c-f3d82291539f service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:30.761466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d40f-26c3-498d-a20c-f3d82291539f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:30.763509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4d72d40f-26c3-498d-a20c-f3d82291539f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:30.873098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72d40f-26c3-498d-a20c-f3d82291539f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:30.874493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72d40f-26c3-498d-a20c-f3d82291539f service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:30.893086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 727/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:33.066312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1326cc-9e6b-45a3-8a54-b5ce0b86b30b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:33.066877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1326cc-9e6b-45a3-8a54-b5ce0b86b30b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:33.066877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1326cc-9e6b-45a3-8a54-b5ce0b86b30b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:33.068257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 761/1488] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:33.073807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f5aff-9116-4efd-8df4-d142cb055e21 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:33.074201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f5aff-9116-4efd-8df4-d142cb055e21 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:33.074367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f5aff-9116-4efd-8df4-d142cb055e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:33.139941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3a9f5aff-9116-4efd-8df4-d142cb055e21 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:33.141136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3a9f5aff-9116-4efd-8df4-d142cb055e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:33.196704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:33.242227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 728/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:33.284748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1326cc-9e6b-45a3-8a54-b5ce0b86b30b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:33.286367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4b1326cc-9e6b-45a3-8a54-b5ce0b86b30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:33.321678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:33.388425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1326cc-9e6b-45a3-8a54-b5ce0b86b30b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:33.461808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17e00-42bf-46a0-a918-a5807f414b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:33.462426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17e00-42bf-46a0-a918-a5807f414b14 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:33.462705 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17e00-42bf-46a0-a918-a5807f414b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:33.462975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17e00-42bf-46a0-a918-a5807f414b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:33.464583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-76c17e00-42bf-46a0-a918-a5807f414b14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:33.544242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:33.574622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c17e00-42bf-46a0-a918-a5807f414b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:33.575991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c17e00-42bf-46a0-a918-a5807f414b14 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:33.594942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 762/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:35.830681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3a56-3e0e-4b6b-afde-0734005eca88 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:35.831256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3a56-3e0e-4b6b-afde-0734005eca88 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:35.831256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3a56-3e0e-4b6b-afde-0734005eca88 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:35.832616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 729/1491] 10.222.0.20 () {54 vars in 926 bytes} [Thu Jul 28 06:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:31:35.837739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a326c8-db8c-4097-a07b-30df783bbe2a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:35.838134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a326c8-db8c-4097-a07b-30df783bbe2a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:35.838286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a326c8-db8c-4097-a07b-30df783bbe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:35.903879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d2a326c8-db8c-4097-a07b-30df783bbe2a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:31:35.905068 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d2a326c8-db8c-4097-a07b-30df783bbe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:36.002449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 763/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Thu Jul 28 06:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:31:36.041525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e3a56-3e0e-4b6b-afde-0734005eca88 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:31:36.043077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d78e3a56-3e0e-4b6b-afde-0734005eca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:36.147070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e3a56-3e0e-4b6b-afde-0734005eca88 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:31:36.225672 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a542cbf-cd5d-4579-b63b-612ade2e6502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:31:36.226290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a542cbf-cd5d-4579-b63b-612ade2e6502 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:31:36.226574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a542cbf-cd5d-4579-b63b-612ade2e6502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:31:36.226850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a542cbf-cd5d-4579-b63b-612ade2e6502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:31:36.228473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0a542cbf-cd5d-4579-b63b-612ade2e6502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:31:36.237711 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:36.335890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a542cbf-cd5d-4579-b63b-612ade2e6502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:31:36.337337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a542cbf-cd5d-4579-b63b-612ade2e6502 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:31:36.355456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 730/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:31:46.000647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=82 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:31:46.353230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=50 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:04.883176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0790-4383-4aff-8c1a-bb11b3231da4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:04.883784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0790-4383-4aff-8c1a-bb11b3231da4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:04.883784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0790-4383-4aff-8c1a-bb11b3231da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:04.961474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-aefb0790-4383-4aff-8c1a-bb11b3231da4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:33:04.962680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aefb0790-4383-4aff-8c1a-bb11b3231da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:05.059216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 764/1494] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:33:05.067517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a037923-de22-4ad9-b1f0-9f87ba1775cb None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:05.068013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a037923-de22-4ad9-b1f0-9f87ba1775cb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:05.068128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a037923-de22-4ad9-b1f0-9f87ba1775cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:05.157944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2a037923-de22-4ad9-b1f0-9f87ba1775cb None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:33:05.159347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2a037923-de22-4ad9-b1f0-9f87ba1775cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:05.264934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 731/1495] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:33:05.270260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb0790-4383-4aff-8c1a-bb11b3231da4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:05.271516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aefb0790-4383-4aff-8c1a-bb11b3231da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:05.283086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb0790-4383-4aff-8c1a-bb11b3231da4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:05.345874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:05.380707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:05.381342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:05.381630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:05.381908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:05.382604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:33:05.382911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:05.383953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:33:05.383953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 28 06:33:05.383953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: Jul 28 06:33:05.384836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:05.395771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 765/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:33:05.404357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a037923-de22-4ad9-b1f0-9f87ba1775cb None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:05.414092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a037923-de22-4ad9-b1f0-9f87ba1775cb None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:05.478006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:05.516265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:05.516877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:05.517154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:05.517454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:05.518112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:05.519834 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:33:05.519834 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 28 06:33:05.519834 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: Jul 28 06:33:05.521032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:05.580949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 732/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:33:05 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:33:05.585263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:05.586806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:05.619572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ce41-d5f1-460c-9918-b78e4cb2d629 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:05.716983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:05.717555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:05.717840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:05.718114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:05.719334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:05.720342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:33:05.720342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 28 06:33:05.720342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: Jul 28 06:33:05.721578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:05.762532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 766/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:33:05 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:33:05.766745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:05.768277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:05.831186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7cdfc-c6ab-4b44-8784-aa7061ef9955 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:05.917162 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:05.929054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:05.929719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:05.930006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:05.930284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:05.930981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:05.931999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:33:05.931999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 28 06:33:05.931999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: Jul 28 06:33:05.932888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:05.945270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 733/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:33:05.949596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:05.951109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:05.962679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14c907-60fd-4430-b70a-67513bb41fbb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:05.988264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:06.057816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153eee2-d999-4018-9d28-5602d296f444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:06.058429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153eee2-d999-4018-9d28-5602d296f444 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:06.058710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153eee2-d999-4018-9d28-5602d296f444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:06.058982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153eee2-d999-4018-9d28-5602d296f444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:06.059680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153eee2-d999-4018-9d28-5602d296f444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:06.060696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-d153eee2-d999-4018-9d28-5602d296f444 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:33:06.060696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 28 06:33:06.060696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: Jul 28 06:33:06.061604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153eee2-d999-4018-9d28-5602d296f444 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:06.074115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 767/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:33:06.079078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:06.080765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:06.113803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4b36a-496f-44d7-80aa-96db9d06894f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:06.213872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8fca4-4b63-4b5f-a104-d729b4988d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:06.214542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fca4-4b63-4b5f-a104-d729b4988d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:06.214824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fca4-4b63-4b5f-a104-d729b4988d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:06.215098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fca4-4b63-4b5f-a104-d729b4988d49 admin admin] PATH_INFO: `/v3/projects/b37af19fd29b4000b816d3c06b248032/users/715c0ab08d8f4a979a80bd6245698b6c/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:06.237189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8fca4-4b63-4b5f-a104-d729b4988d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37af19fd29b4000b816d3c06b248032, user_id=715c0ab08d8f4a979a80bd6245698b6c, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:06.238239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-f1a8fca4-4b63-4b5f-a104-d729b4988d49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:33:06.238239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 28 06:33:06.238239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: Jul 28 06:33:06.240331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8fca4-4b63-4b5f-a104-d729b4988d49 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:06.264120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 734/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:33:06 2022] PUT /identity/v3/projects/b37af19fd29b4000b816d3c06b248032/users/715c0ab08d8f4a979a80bd6245698b6c/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:33:06.277109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da59fe-ddce-481a-b72b-1e1c78c8b3b0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:06.277430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da59fe-ddce-481a-b72b-1e1c78c8b3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:06.277710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da59fe-ddce-481a-b72b-1e1c78c8b3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:06.335158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-04da59fe-ddce-481a-b72b-1e1c78c8b3b0 None None] MFA Rules not processed for user `715c0ab08d8f4a979a80bd6245698b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:33:06.336361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-04da59fe-ddce-481a-b72b-1e1c78c8b3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:06.412627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 768/1502] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:33:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:33:06.425768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0337d-ee5d-4d54-ae29-a561a5791f58 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:06.426082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0337d-ee5d-4d54-ae29-a561a5791f58 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:06.426361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0337d-ee5d-4d54-ae29-a561a5791f58 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:06.427692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 735/1503] 10.222.0.20 () {54 vars in 964 bytes} [Thu Jul 28 06:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:33:06.432629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349f386-c860-41e3-9462-2499562f0dfb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:06.432925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349f386-c860-41e3-9462-2499562f0dfb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:06.433207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349f386-c860-41e3-9462-2499562f0dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:06.466152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:06.510748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e349f386-c860-41e3-9462-2499562f0dfb None None] MFA Rules not processed for user `d6ee12d4109f4d8ba0d502558b9a2ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:33:06.511877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e349f386-c860-41e3-9462-2499562f0dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:06.590598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 769/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Thu Jul 28 06:33:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:33:06.598551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0337d-ee5d-4d54-ae29-a561a5791f58 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:06.600088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-75d0337d-ee5d-4d54-ae29-a561a5791f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:06.651404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:06.707478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0337d-ee5d-4d54-ae29-a561a5791f58 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:06.780400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:06.780938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:06.781229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:06.782183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:06.783140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:06.881457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:06.882890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:06.886395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:06.900495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 736/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:33:06.949275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62827eac-8b36-4614-8e3a-84c81c4d8e30 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:06.949594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62827eac-8b36-4614-8e3a-84c81c4d8e30 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:06.949873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62827eac-8b36-4614-8e3a-84c81c4d8e30 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:06.951136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 770/1506] 10.222.0.20 () {52 vars in 830 bytes} [Thu Jul 28 06:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:33:06.954823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:06.956020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:07.053618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ae664-7f73-4f47-9909-347e9f5ab381 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:07.148095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:07.148773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:07.149008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:07.149284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] PATH_INFO: `/v3/users/715c0ab08d8f4a979a80bd6245698b6c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:07.161699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=715c0ab08d8f4a979a80bd6245698b6c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:07.162719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:33:07.162719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 28 06:33:07.162719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: Jul 28 06:33:07.163931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:07.174449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:07.231336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715c0ab08d8f4a979a80bd6245698b6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:07.243547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715c0ab08d8f4a979a80bd6245698b6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:07.244180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715c0ab08d8f4a979a80bd6245698b6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:07.262252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715c0ab08d8f4a979a80bd6245698b6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:07.262679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715c0ab08d8f4a979a80bd6245698b6c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:07.280507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 737/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:33:06 2022] DELETE /identity/v3/users/715c0ab08d8f4a979a80bd6245698b6c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:33:07.285730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5389-e06f-4466-9ccd-cc356cd1e1ba None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:07.286040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5389-e06f-4466-9ccd-cc356cd1e1ba None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:07.286315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5389-e06f-4466-9ccd-cc356cd1e1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:07.341727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-32dd5389-e06f-4466-9ccd-cc356cd1e1ba None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:33:07.342853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-32dd5389-e06f-4466-9ccd-cc356cd1e1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:07.418935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:07.433463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 771/1508] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:33:07.443712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:07.444985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:07.547753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25167bdc-71b8-4852-80da-55a167a190f0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:07.603129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:07.626262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d975a0a-d5fe-4746-b4fc-93257875724c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:07.626910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975a0a-d5fe-4746-b4fc-93257875724c service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:07.627189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975a0a-d5fe-4746-b4fc-93257875724c service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:07.627465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975a0a-d5fe-4746-b4fc-93257875724c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:07.629180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6d975a0a-d5fe-4746-b4fc-93257875724c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:07.747804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d975a0a-d5fe-4746-b4fc-93257875724c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:07.749246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d975a0a-d5fe-4746-b4fc-93257875724c service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:07.768218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 738/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:33:08.098189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd5389-e06f-4466-9ccd-cc356cd1e1ba None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:33:08.099473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-32dd5389-e06f-4466-9ccd-cc356cd1e1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:33:08.111077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd5389-e06f-4466-9ccd-cc356cd1e1ba None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:33:08.216329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:33:08.216978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:33:08.217258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:33:08.217564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] PATH_INFO: `/v3/projects/b37af19fd29b4000b816d3c06b248032` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:33:08.227466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37af19fd29b4000b816d3c06b248032)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:33:08.228489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:33:08.228489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 28 06:33:08.228489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: Jul 28 06:33:08.229756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:33:08.288460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37af19fd29b4000b816d3c06b248032', 'request_id': 'req-2ca11e05-db47-4803-b3de-e718115c3288'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:08.294715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37af19fd29b4000b816d3c06b248032', 'request_id': 'req-2ca11e05-db47-4803-b3de-e718115c3288'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:08.304846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37af19fd29b4000b816d3c06b248032', 'request_id': 'req-2ca11e05-db47-4803-b3de-e718115c3288'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:08.334633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] The token cache is being invalidate because project b37af19fd29b4000b816d3c06b248032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:33:08.334925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:08.335441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2ca11e05-db47-4803-b3de-e718115c3288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:33:08.336760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 772/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:33:08 2022] DELETE /identity/v3/projects/b37af19fd29b4000b816d3c06b248032 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:33:17.765859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=62 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:33:18.332648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:51.424229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d56e1d-e6b8-4d2c-b222-3930f107823a None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:51.424878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d56e1d-e6b8-4d2c-b222-3930f107823a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:51.424878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d56e1d-e6b8-4d2c-b222-3930f107823a None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:51.426195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 739/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:34:51.430951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fce8e7-bb17-4bf1-aac2-ae2f2cd7b406 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:51.431374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fce8e7-bb17-4bf1-aac2-ae2f2cd7b406 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:51.431527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fce8e7-bb17-4bf1-aac2-ae2f2cd7b406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:51.515188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-93fce8e7-bb17-4bf1-aac2-ae2f2cd7b406 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:51.516434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-93fce8e7-bb17-4bf1-aac2-ae2f2cd7b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:51.621145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 773/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:51.777716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d56e1d-e6b8-4d2c-b222-3930f107823a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:51.779796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-60d56e1d-e6b8-4d2c-b222-3930f107823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:51.900154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d56e1d-e6b8-4d2c-b222-3930f107823a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:51.905676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b669d7-7288-495a-8806-1d355c204857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:51.906292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b669d7-7288-495a-8806-1d355c204857 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:51.906573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b669d7-7288-495a-8806-1d355c204857 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:51.906848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b669d7-7288-495a-8806-1d355c204857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:51.914849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b669d7-7288-495a-8806-1d355c204857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:51.916475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b669d7-7288-495a-8806-1d355c204857 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:51.926302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 740/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:52.763797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91ab4-75df-498f-8877-aa71a31a2641 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.764329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91ab4-75df-498f-8877-aa71a31a2641 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.764329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91ab4-75df-498f-8877-aa71a31a2641 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.765757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 774/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:34:52.771377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74dd2a-39f7-46d0-abbc-31bb4720984b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.771797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74dd2a-39f7-46d0-abbc-31bb4720984b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.771941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74dd2a-39f7-46d0-abbc-31bb4720984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.793102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cd74dd2a-39f7-46d0-abbc-31bb4720984b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:52.794337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cd74dd2a-39f7-46d0-abbc-31bb4720984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:52.802287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 741/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:34:52.954745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c91ab4-75df-498f-8877-aa71a31a2641 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:34:52.969947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c91ab4-75df-498f-8877-aa71a31a2641 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:34:52.976004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9fcc7-a34f-4f4f-9e95-5c1d006b0ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:34:52.976640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9fcc7-a34f-4f4f-9e95-5c1d006b0ded service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:52.976934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9fcc7-a34f-4f4f-9e95-5c1d006b0ded service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:52.977192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9fcc7-a34f-4f4f-9e95-5c1d006b0ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:52.985118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9fcc7-a34f-4f4f-9e95-5c1d006b0ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:34:52.986716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9fcc7-a34f-4f4f-9e95-5c1d006b0ded service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:34:52.996588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 775/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:34:55.304455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76293fba-71fb-4870-93aa-1fc7047902cb None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:55.305063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76293fba-71fb-4870-93aa-1fc7047902cb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:55.305063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76293fba-71fb-4870-93aa-1fc7047902cb None None] PATH_INFO: `/v3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:55.314229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 742/1517] 10.222.0.49 () {54 vars in 925 bytes} [Thu Jul 28 06:34:55 2022] GET /identity/v3 => generated 255 bytes in 11 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:34:55.323996 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9db31-7bd3-4360-a6cd-cb9f81d96753 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:34:55.324465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9db31-7bd3-4360-a6cd-cb9f81d96753 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:34:55.324571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9db31-7bd3-4360-a6cd-cb9f81d96753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:34:55.384552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-59d9db31-7bd3-4360-a6cd-cb9f81d96753 None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:34:55.385819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-59d9db31-7bd3-4360-a6cd-cb9f81d96753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:34:55.402842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:34:55.459177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 776/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Thu Jul 28 06:34:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:02.788347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=56 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:04.237070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a7694-3743-4734-b66a-3b35d25032f0 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:04.237768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a7694-3743-4734-b66a-3b35d25032f0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:04.237768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a7694-3743-4734-b66a-3b35d25032f0 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:04.239039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 743/1519] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:35:04.244124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1e1bd-9716-43dd-a694-4af47ae40208 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:04.244570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1e1bd-9716-43dd-a694-4af47ae40208 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:04.244679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1e1bd-9716-43dd-a694-4af47ae40208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:04.264566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-51e1e1bd-9716-43dd-a694-4af47ae40208 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:04.265855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-51e1e1bd-9716-43dd-a694-4af47ae40208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:04.275325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 777/1520] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:04.429134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a7694-3743-4734-b66a-3b35d25032f0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:04.444582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a7694-3743-4734-b66a-3b35d25032f0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:04.450395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5fab0-78d9-460c-b3d7-81aa44a53c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:04.451013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5fab0-78d9-460c-b3d7-81aa44a53c20 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:04.451298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5fab0-78d9-460c-b3d7-81aa44a53c20 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:04.451576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5fab0-78d9-460c-b3d7-81aa44a53c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:04.460243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5fab0-78d9-460c-b3d7-81aa44a53c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:04.461854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5fab0-78d9-460c-b3d7-81aa44a53c20 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:04.471331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 744/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:05.285214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2529cb-1f28-461e-af69-061506d5c393 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:05.285843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2529cb-1f28-461e-af69-061506d5c393 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:05.285843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2529cb-1f28-461e-af69-061506d5c393 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:05.288223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 778/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:35:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:35:05.292934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e68f-dea4-40e0-a0a7-c348c218c04c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:05.293399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e68f-dea4-40e0-a0a7-c348c218c04c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:05.293511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e68f-dea4-40e0-a0a7-c348c218c04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:05.312414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1720e68f-dea4-40e0-a0a7-c348c218c04c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:05.313650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1720e68f-dea4-40e0-a0a7-c348c218c04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:05.322023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 745/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:05.476601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2529cb-1f28-461e-af69-061506d5c393 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:05.491544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2529cb-1f28-461e-af69-061506d5c393 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:05.496937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc10d92-1760-4929-b05c-66802fadbbe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:05.497575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10d92-1760-4929-b05c-66802fadbbe7 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:05.497856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10d92-1760-4929-b05c-66802fadbbe7 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:05.498128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10d92-1760-4929-b05c-66802fadbbe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:05.507026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc10d92-1760-4929-b05c-66802fadbbe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:05.508591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc10d92-1760-4929-b05c-66802fadbbe7 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:05.517913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 779/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:06.943002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa037959-269e-42bf-826c-cb4044d8fff3 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:06.943570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa037959-269e-42bf-826c-cb4044d8fff3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:06.943570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa037959-269e-42bf-826c-cb4044d8fff3 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:06.944954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 746/1525] 10.222.0.20 () {54 vars in 961 bytes} [Thu Jul 28 06:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:35:06.950393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ab9ac-b86b-4e0a-8189-d2c1830bf71d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:06.950810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ab9ac-b86b-4e0a-8189-d2c1830bf71d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:06.950954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ab9ac-b86b-4e0a-8189-d2c1830bf71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:06.970408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-de8ab9ac-b86b-4e0a-8189-d2c1830bf71d None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:06.971602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-de8ab9ac-b86b-4e0a-8189-d2c1830bf71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:06.980896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 780/1526] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Jul 28 06:35:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:06.988236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa037959-269e-42bf-826c-cb4044d8fff3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:07.007827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa037959-269e-42bf-826c-cb4044d8fff3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:07.012737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda19a4-f595-48ff-b0f4-2256c0307ad9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:07.013366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda19a4-f595-48ff-b0f4-2256c0307ad9 service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:07.013649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda19a4-f595-48ff-b0f4-2256c0307ad9 service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:07.013920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda19a4-f595-48ff-b0f4-2256c0307ad9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:07.023386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda19a4-f595-48ff-b0f4-2256c0307ad9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:07.024946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda19a4-f595-48ff-b0f4-2256c0307ad9 service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:07.034375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 747/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:16.746373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627665f-e8a7-4320-bb5f-1fe5f6844f17 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:16.746922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627665f-e8a7-4320-bb5f-1fe5f6844f17 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:16.746922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627665f-e8a7-4320-bb5f-1fe5f6844f17 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:16.748288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 781/1528] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:35:16.753102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a14cf3-ef7f-4e0d-b98a-e7220016e9c3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:16.753551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a14cf3-ef7f-4e0d-b98a-e7220016e9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:16.753685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a14cf3-ef7f-4e0d-b98a-e7220016e9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:16.772951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e4a14cf3-ef7f-4e0d-b98a-e7220016e9c3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:16.774180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e4a14cf3-ef7f-4e0d-b98a-e7220016e9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:16.782205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 748/1529] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:16.937999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627665f-e8a7-4320-bb5f-1fe5f6844f17 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:16.953246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627665f-e8a7-4320-bb5f-1fe5f6844f17 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:16.959609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbe9ad-1bbe-4867-8acc-fcc74473579e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:16.960218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbe9ad-1bbe-4867-8acc-fcc74473579e service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:16.960502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbe9ad-1bbe-4867-8acc-fcc74473579e service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:16.960778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbe9ad-1bbe-4867-8acc-fcc74473579e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:16.968991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbe9ad-1bbe-4867-8acc-fcc74473579e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:16.970604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbe9ad-1bbe-4867-8acc-fcc74473579e service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:16.980890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 782/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:17.804330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce86f5f-4453-4cb0-81b5-05b9c09341e9 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:17.804872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce86f5f-4453-4cb0-81b5-05b9c09341e9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:17.804872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce86f5f-4453-4cb0-81b5-05b9c09341e9 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:17.806257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 749/1531] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 06:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:35:17.811082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b927bb-5f8b-4c92-9cb0-5c4138775cb5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:17.811494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b927bb-5f8b-4c92-9cb0-5c4138775cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:17.811640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b927bb-5f8b-4c92-9cb0-5c4138775cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:17.831822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a0b927bb-5f8b-4c92-9cb0-5c4138775cb5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:17.833020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a0b927bb-5f8b-4c92-9cb0-5c4138775cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:17.842046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 783/1532] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 06:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:17.992388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce86f5f-4453-4cb0-81b5-05b9c09341e9 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:18.008035 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce86f5f-4453-4cb0-81b5-05b9c09341e9 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:18.013459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe366f-31b6-46f6-a4f5-2799a5df6e9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:18.014067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe366f-31b6-46f6-a4f5-2799a5df6e9c service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:18.014347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe366f-31b6-46f6-a4f5-2799a5df6e9c service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:18.014630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe366f-31b6-46f6-a4f5-2799a5df6e9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:18.022949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe366f-31b6-46f6-a4f5-2799a5df6e9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:18.024517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe366f-31b6-46f6-a4f5-2799a5df6e9c service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:18.033933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 750/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:21.939427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d87841-fa73-42ea-b84f-88ab42b7b860 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:21.939999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d87841-fa73-42ea-b84f-88ab42b7b860 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:21.939999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d87841-fa73-42ea-b84f-88ab42b7b860 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:21.941395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 784/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:35:21.947902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd6f2-67e9-49d9-8f95-be42df7ac6e4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:21.948334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd6f2-67e9-49d9-8f95-be42df7ac6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:21.948433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd6f2-67e9-49d9-8f95-be42df7ac6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:21.967749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-932bd6f2-67e9-49d9-8f95-be42df7ac6e4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:21.968921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-932bd6f2-67e9-49d9-8f95-be42df7ac6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:21.976860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 751/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:22.049501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d87841-fa73-42ea-b84f-88ab42b7b860 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:22.051206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c6d87841-fa73-42ea-b84f-88ab42b7b860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:22.129556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d87841-fa73-42ea-b84f-88ab42b7b860 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:22.134223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb7307-989f-4bef-b4a4-107a3af1bccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:22.134838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb7307-989f-4bef-b4a4-107a3af1bccd service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:22.135118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb7307-989f-4bef-b4a4-107a3af1bccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:22.135390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb7307-989f-4bef-b4a4-107a3af1bccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:22.144190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb7307-989f-4bef-b4a4-107a3af1bccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:22.145647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb7307-989f-4bef-b4a4-107a3af1bccd service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:22.156009 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 785/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:24.178022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d2a9-da85-4252-916e-48ee09a5cee3 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:24.178595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d2a9-da85-4252-916e-48ee09a5cee3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:24.178595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d2a9-da85-4252-916e-48ee09a5cee3 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:24.179941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 752/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:35:24.185143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e86c8-1acf-4ea1-810f-2b30b2ae191d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:24.185602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e86c8-1acf-4ea1-810f-2b30b2ae191d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:24.185733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e86c8-1acf-4ea1-810f-2b30b2ae191d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:24.207324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9a1e86c8-1acf-4ea1-810f-2b30b2ae191d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:24.209132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9a1e86c8-1acf-4ea1-810f-2b30b2ae191d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:24.216937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 786/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:24.253843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d2a9-da85-4252-916e-48ee09a5cee3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:24.264809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d2a9-da85-4252-916e-48ee09a5cee3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:24.268243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fe8a7-283f-45db-a4f2-295f584413e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:24.268847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fe8a7-283f-45db-a4f2-295f584413e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:24.269125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fe8a7-283f-45db-a4f2-295f584413e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:24.269431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fe8a7-283f-45db-a4f2-295f584413e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:24.278353 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fe8a7-283f-45db-a4f2-295f584413e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:24.279759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fe8a7-283f-45db-a4f2-295f584413e2 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:24.289650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 753/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:27.791718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccba83b-8702-4d34-8a5a-67c4d87131c8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:27.792280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccba83b-8702-4d34-8a5a-67c4d87131c8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:27.792280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccba83b-8702-4d34-8a5a-67c4d87131c8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:27.793677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 787/1540] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:35:27.798785 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b3d6-a08c-4a0b-b2a0-a2680502c68d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:27.799328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b3d6-a08c-4a0b-b2a0-a2680502c68d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:27.799328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b3d6-a08c-4a0b-b2a0-a2680502c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:27.819562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c328b3d6-a08c-4a0b-b2a0-a2680502c68d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:27.821012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c328b3d6-a08c-4a0b-b2a0-a2680502c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:27.828759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 754/1541] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:28.130740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccba83b-8702-4d34-8a5a-67c4d87131c8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:28.132471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2ccba83b-8702-4d34-8a5a-67c4d87131c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:28.136541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=83 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:28.213910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccba83b-8702-4d34-8a5a-67c4d87131c8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:28.217524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bcb35-d923-4f35-8cc2-557929b80c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:28.218142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bcb35-d923-4f35-8cc2-557929b80c64 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:28.218425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bcb35-d923-4f35-8cc2-557929b80c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:28.218703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bcb35-d923-4f35-8cc2-557929b80c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:28.228249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bcb35-d923-4f35-8cc2-557929b80c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:28.229700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bcb35-d923-4f35-8cc2-557929b80c64 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:28.238907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 788/1542] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:30.376949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc6181-2ae2-4361-ab3f-b31e7df4defe None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:30.377503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc6181-2ae2-4361-ab3f-b31e7df4defe None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:30.377503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc6181-2ae2-4361-ab3f-b31e7df4defe None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:30.378868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 755/1543] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 06:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:35:30.385237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aee07-f28b-4b37-abcc-a14b6e7700b9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:30.385713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aee07-f28b-4b37-abcc-a14b6e7700b9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:30.385879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aee07-f28b-4b37-abcc-a14b6e7700b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:30.405832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-620aee07-f28b-4b37-abcc-a14b6e7700b9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:35:30.407064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-620aee07-f28b-4b37-abcc-a14b6e7700b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:30.415440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 789/1544] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 06:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:35:30.725644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc6181-2ae2-4361-ab3f-b31e7df4defe None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:35:30.727454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-72cc6181-2ae2-4361-ab3f-b31e7df4defe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:35:30.745708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc6181-2ae2-4361-ab3f-b31e7df4defe None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:35:30.749268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16031c9-8b74-4621-8fa2-ba2c37868d38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:35:30.749915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16031c9-8b74-4621-8fa2-ba2c37868d38 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:35:30.750200 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16031c9-8b74-4621-8fa2-ba2c37868d38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:35:30.750479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16031c9-8b74-4621-8fa2-ba2c37868d38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:35:30.760520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16031c9-8b74-4621-8fa2-ba2c37868d38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:35:30.762004 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16031c9-8b74-4621-8fa2-ba2c37868d38 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:35:30.771311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 756/1545] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:35:40.400564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=37 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:35:40.767204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=42 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:01.878739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9ca5-8e35-4211-942a-1a76762f64a5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.880845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9ca5-8e35-4211-942a-1a76762f64a5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.880845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9ca5-8e35-4211-942a-1a76762f64a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.881831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d66e9-4d94-4cf3-8502-b6c929f699fd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.882433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d66e9-4d94-4cf3-8502-b6c929f699fd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.882433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d66e9-4d94-4cf3-8502-b6c929f699fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.911330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-10bc9ca5-8e35-4211-942a-1a76762f64a5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:01.913473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-745d66e9-4d94-4cf3-8502-b6c929f699fd None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:01.914388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-10bc9ca5-8e35-4211-942a-1a76762f64a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:01.914915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-745d66e9-4d94-4cf3-8502-b6c929f699fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:01.930463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 790/1546] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:01.931029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 757/1547] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:01.939579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da600c26-e0a4-42d7-b6bd-96a865507aa1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.940258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6072126-dfcc-4e40-af6c-50459f176a02 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:01.940660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6072126-dfcc-4e40-af6c-50459f176a02 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.940796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da600c26-e0a4-42d7-b6bd-96a865507aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:01.941137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6072126-dfcc-4e40-af6c-50459f176a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.941483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da600c26-e0a4-42d7-b6bd-96a865507aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:01.972313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d6072126-dfcc-4e40-af6c-50459f176a02 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:01.974315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d6072126-dfcc-4e40-af6c-50459f176a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:01.975286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-da600c26-e0a4-42d7-b6bd-96a865507aa1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:01.976663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-da600c26-e0a4-42d7-b6bd-96a865507aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:01.991113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 758/1548] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:01.992924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 791/1549] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:01.995826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6072126-dfcc-4e40-af6c-50459f176a02 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:01.997730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da600c26-e0a4-42d7-b6bd-96a865507aa1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.005788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6072126-dfcc-4e40-af6c-50459f176a02 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.018021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9980e4a-a907-486a-8311-b3900c6a6d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.018352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da600c26-e0a4-42d7-b6bd-96a865507aa1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.018798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980e4a-a907-486a-8311-b3900c6a6d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.019153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980e4a-a907-486a-8311-b3900c6a6d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.019502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980e4a-a907-486a-8311-b3900c6a6d3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.020375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9980e4a-a907-486a-8311-b3900c6a6d3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.020756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9980e4a-a907-486a-8311-b3900c6a6d3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.023310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-e9980e4a-a907-486a-8311-b3900c6a6d3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:37:02.023310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 28 06:37:02.023310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: Jul 28 06:37:02.024374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9980e4a-a907-486a-8311-b3900c6a6d3b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.030597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.031527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.032166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.032652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.033601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.034124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.036235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.045816 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 759/1550] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:02.047106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d354-8e53-489a-a166-370991acca5f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.047394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d354-8e53-489a-a166-370991acca5f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.054508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d354-8e53-489a-a166-370991acca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.066825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 792/1551] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:37:02.068506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.087341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ba79d354-8e53-489a-a166-370991acca5f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.088575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ba79d354-8e53-489a-a166-370991acca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.090482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdc684-1ac6-4dc4-8ce6-b0c6a3de706a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.103023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd46d5-91e9-4c26-98d2-8e29d71e8c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.103237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 760/1552] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.103984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79d354-8e53-489a-a166-370991acca5f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.104495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd46d5-91e9-4c26-98d2-8e29d71e8c69 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.105134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd46d5-91e9-4c26-98d2-8e29d71e8c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.105656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd46d5-91e9-4c26-98d2-8e29d71e8c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.106607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd46d5-91e9-4c26-98d2-8e29d71e8c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.107846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-c0bd46d5-91e9-4c26-98d2-8e29d71e8c69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:37:02.107846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 28 06:37:02.107846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: Jul 28 06:37:02.109469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd46d5-91e9-4c26-98d2-8e29d71e8c69 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.118476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79d354-8e53-489a-a166-370991acca5f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.124097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc393d0f-f61d-46ab-b714-c1e853f334fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.124735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393d0f-f61d-46ab-b714-c1e853f334fb admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.125018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393d0f-f61d-46ab-b714-c1e853f334fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.125309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393d0f-f61d-46ab-b714-c1e853f334fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.126451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc393d0f-f61d-46ab-b714-c1e853f334fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.131162 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc393d0f-f61d-46ab-b714-c1e853f334fb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.169673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 761/1553] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.171138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465cb3-6145-403f-ab13-5ad2f4ad7e79 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.171283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 793/1554] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/projects => generated 391 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.171605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465cb3-6145-403f-ab13-5ad2f4ad7e79 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.171894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465cb3-6145-403f-ab13-5ad2f4ad7e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.172579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f52eb-d3b6-4d5c-9092-51d8e09bfb04 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.172871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f52eb-d3b6-4d5c-9092-51d8e09bfb04 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.173176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f52eb-d3b6-4d5c-9092-51d8e09bfb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.201799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b81f52eb-d3b6-4d5c-9092-51d8e09bfb04 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.203180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b81f52eb-d3b6-4d5c-9092-51d8e09bfb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.205058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e0465cb3-6145-403f-ab13-5ad2f4ad7e79 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.206230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e0465cb3-6145-403f-ab13-5ad2f4ad7e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.264911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 794/1555] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.266203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3756b-5127-41b6-ab0f-2ca07d533ef0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.266492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3756b-5127-41b6-ab0f-2ca07d533ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.266766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3756b-5127-41b6-ab0f-2ca07d533ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.276242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 762/1556] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.277706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64e449-9596-4a88-891d-a4e76caf249c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.277999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64e449-9596-4a88-891d-a4e76caf249c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.278278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64e449-9596-4a88-891d-a4e76caf249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.287828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d9c3756b-5127-41b6-ab0f-2ca07d533ef0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.289039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d9c3756b-5127-41b6-ab0f-2ca07d533ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.297089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 795/1557] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.297872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3756b-5127-41b6-ab0f-2ca07d533ef0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.299931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1b64e449-9596-4a88-891d-a4e76caf249c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.301343 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1b64e449-9596-4a88-891d-a4e76caf249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.307554 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3756b-5127-41b6-ab0f-2ca07d533ef0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.309277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 763/1558] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.310112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64e449-9596-4a88-891d-a4e76caf249c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.312262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ae8e4-e127-42da-b494-86f541a72b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.312899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ae8e4-e127-42da-b494-86f541a72b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.313176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ae8e4-e127-42da-b494-86f541a72b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.313507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ae8e4-e127-42da-b494-86f541a72b5e admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.314167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ae8e4-e127-42da-b494-86f541a72b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.316056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ae8e4-e127-42da-b494-86f541a72b5e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.326996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64e449-9596-4a88-891d-a4e76caf249c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.331612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c9dd8-ab73-40fe-95a0-6dd1615e3891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.332219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c9dd8-ab73-40fe-95a0-6dd1615e3891 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.332508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c9dd8-ab73-40fe-95a0-6dd1615e3891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.332809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c9dd8-ab73-40fe-95a0-6dd1615e3891 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.333488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c9dd8-ab73-40fe-95a0-6dd1615e3891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.335681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-764c9dd8-ab73-40fe-95a0-6dd1615e3891 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:37:02.335681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 28 06:37:02.335681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: Jul 28 06:37:02.336873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c9dd8-ab73-40fe-95a0-6dd1615e3891 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.340277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 796/1559] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.342714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d5b3-8b93-4e16-836b-5d4a0c933358 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.343461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d5b3-8b93-4e16-836b-5d4a0c933358 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.343633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d5b3-8b93-4e16-836b-5d4a0c933358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.360760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 764/1560] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/users => generated 296 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.360760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfdc78-1515-4d69-ae6b-f85cec511e68 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.360760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfdc78-1515-4d69-ae6b-f85cec511e68 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.360760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfdc78-1515-4d69-ae6b-f85cec511e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.363461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9fb0d5b3-8b93-4e16-836b-5d4a0c933358 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.364671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9fb0d5b3-8b93-4e16-836b-5d4a0c933358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.372672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 797/1561] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.373962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871047ef-a95c-437a-b16f-983f4614e1f2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.374255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871047ef-a95c-437a-b16f-983f4614e1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.374535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871047ef-a95c-437a-b16f-983f4614e1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.381746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-16dfdc78-1515-4d69-ae6b-f85cec511e68 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.383176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-16dfdc78-1515-4d69-ae6b-f85cec511e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.392646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 765/1562] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.393872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f637c6-ac45-4c3f-b3e7-c3f84309df3d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.393990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-871047ef-a95c-437a-b16f-983f4614e1f2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.394165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f637c6-ac45-4c3f-b3e7-c3f84309df3d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.394431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f637c6-ac45-4c3f-b3e7-c3f84309df3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.395141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-871047ef-a95c-437a-b16f-983f4614e1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.409075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 798/1563] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.409949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871047ef-a95c-437a-b16f-983f4614e1f2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.418662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-77f637c6-ac45-4c3f-b3e7-c3f84309df3d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.420055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-77f637c6-ac45-4c3f-b3e7-c3f84309df3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.421441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871047ef-a95c-437a-b16f-983f4614e1f2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.426663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.427774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.428060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.428355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.428855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 766/1564] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.429029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.429373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.432012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.432199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7904b-9f9a-4c78-9d3b-a55c5f08cee6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.432199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7904b-9f9a-4c78-9d3b-a55c5f08cee6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.432199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7904b-9f9a-4c78-9d3b-a55c5f08cee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.453558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 799/1565] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:02.453558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.454981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e1a7904b-9f9a-4c78-9d3b-a55c5f08cee6 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.456179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e1a7904b-9f9a-4c78-9d3b-a55c5f08cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.463592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221a8bc-e9da-4402-b8a9-62e2ef19aaeb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.464347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 767/1566] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.465059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7904b-9f9a-4c78-9d3b-a55c5f08cee6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.469924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ea57b-9ee3-4e33-b0e9-1ab24a5376cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.470558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea57b-9ee3-4e33-b0e9-1ab24a5376cb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.470844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea57b-9ee3-4e33-b0e9-1ab24a5376cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.471123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea57b-9ee3-4e33-b0e9-1ab24a5376cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.471824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ea57b-9ee3-4e33-b0e9-1ab24a5376cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.473613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ea57b-9ee3-4e33-b0e9-1ab24a5376cb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.474272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7904b-9f9a-4c78-9d3b-a55c5f08cee6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.479154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cc2f0-050a-41d0-93c9-7ee28f40afd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.479795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cc2f0-050a-41d0-93c9-7ee28f40afd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.480077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cc2f0-050a-41d0-93c9-7ee28f40afd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.480354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cc2f0-050a-41d0-93c9-7ee28f40afd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.481158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cc2f0-050a-41d0-93c9-7ee28f40afd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.482854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cc2f0-050a-41d0-93c9-7ee28f40afd6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.485633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 800/1567] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:02.487001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913cf0-3157-4a8e-8a65-11ccd371207a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.487293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913cf0-3157-4a8e-8a65-11ccd371207a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.487572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913cf0-3157-4a8e-8a65-11ccd371207a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.497428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 768/1568] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:02.497428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfe664-4252-40a7-a4a2-61fa1deb5d6c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.497428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfe664-4252-40a7-a4a2-61fa1deb5d6c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.497428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfe664-4252-40a7-a4a2-61fa1deb5d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.510178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-77913cf0-3157-4a8e-8a65-11ccd371207a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.511438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-77913cf0-3157-4a8e-8a65-11ccd371207a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.516782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c5cfe664-4252-40a7-a4a2-61fa1deb5d6c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:02.517975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c5cfe664-4252-40a7-a4a2-61fa1deb5d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:02.519732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 801/1569] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.520522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77913cf0-3157-4a8e-8a65-11ccd371207a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.526478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 769/1570] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:02.527319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfe664-4252-40a7-a4a2-61fa1deb5d6c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.533089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77913cf0-3157-4a8e-8a65-11ccd371207a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.536662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfe664-4252-40a7-a4a2-61fa1deb5d6c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.537944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.538550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.538831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.539110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.539906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.540213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.541572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.541826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.542175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.542462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.542740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.543431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.543736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.545340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.554817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 802/1571] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:02.555597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.557484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 770/1572] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:02.558156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.568769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e40b93-0924-456c-a2da-58792925c217 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.573737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af1c91-dbd2-47c4-83e6-e6219893b155 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.573893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.574473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.574763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.575038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.575726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.576032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.577657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.578276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1514c72-5504-4d86-b502-518babe67501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.578879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1514c72-5504-4d86-b502-518babe67501 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.579157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1514c72-5504-4d86-b502-518babe67501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.579423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1514c72-5504-4d86-b502-518babe67501 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.580084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1514c72-5504-4d86-b502-518babe67501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.582098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1514c72-5504-4d86-b502-518babe67501 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.589697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 771/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:02.590497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.602809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffbce5-cac1-4fc5-a1a6-d1b0dea3a602 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.604070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:02.608296 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdbc63-9efc-479b-8f8e-df9fe0282c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.608967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdbc63-9efc-479b-8f8e-df9fe0282c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.609259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdbc63-9efc-479b-8f8e-df9fe0282c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.609569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdbc63-9efc-479b-8f8e-df9fe0282c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.610268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdbc63-9efc-479b-8f8e-df9fe0282c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.614630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 803/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.615303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdbc63-9efc-479b-8f8e-df9fe0282c43 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.615393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1514c72-5504-4d86-b502-518babe67501 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.626299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 772/1575] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:02.626299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdbc63-9efc-479b-8f8e-df9fe0282c43 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.634030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdbc63-9efc-479b-8f8e-df9fe0282c43 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.637499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:02.679600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1514c72-5504-4d86-b502-518babe67501 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.684311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8163d23-d264-4d79-a39e-867fe298294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.684927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163d23-d264-4d79-a39e-867fe298294d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.685202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163d23-d264-4d79-a39e-867fe298294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.685503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163d23-d264-4d79-a39e-867fe298294d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.686192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8163d23-d264-4d79-a39e-867fe298294d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.687989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8163d23-d264-4d79-a39e-867fe298294d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.692435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.693086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.693396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.693676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.694499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.695149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.696641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.698414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 804/1576] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:02.699198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8163d23-d264-4d79-a39e-867fe298294d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.709660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8163d23-d264-4d79-a39e-867fe298294d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.709902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 773/1577] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:02.710458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.719994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.720610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.721050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.721363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.722002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a730952-9ee3-443c-ac95-ad1a8717cf84 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.722139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:02.722346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.723929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.727206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47223e8b-601f-49d2-a1c9-ecefc8df9147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.727838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47223e8b-601f-49d2-a1c9-ecefc8df9147 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.728121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47223e8b-601f-49d2-a1c9-ecefc8df9147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.728399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47223e8b-601f-49d2-a1c9-ecefc8df9147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.729058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47223e8b-601f-49d2-a1c9-ecefc8df9147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.730976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47223e8b-601f-49d2-a1c9-ecefc8df9147 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.737313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 805/1578] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:02.738134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.748360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c1021-dbcb-44a6-9625-90e2497c0d12 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.753252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9359c-fe2f-4f4b-886c-945d0f514b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.753901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9359c-fe2f-4f4b-886c-945d0f514b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.754173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9359c-fe2f-4f4b-886c-945d0f514b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.754444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9359c-fe2f-4f4b-886c-945d0f514b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.755122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9359c-fe2f-4f4b-886c-945d0f514b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.757540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9359c-fe2f-4f4b-886c-945d0f514b09 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.762292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 774/1579] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.763199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47223e8b-601f-49d2-a1c9-ecefc8df9147 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.790354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 806/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.791000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9359c-fe2f-4f4b-886c-945d0f514b09 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.840499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47223e8b-601f-49d2-a1c9-ecefc8df9147 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.854966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9359c-fe2f-4f4b-886c-945d0f514b09 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.859952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f747c9-8234-4d3f-b6f6-0e95cd194e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.860559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f747c9-8234-4d3f-b6f6-0e95cd194e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.860838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f747c9-8234-4d3f-b6f6-0e95cd194e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.861261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f747c9-8234-4d3f-b6f6-0e95cd194e24 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.861943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f747c9-8234-4d3f-b6f6-0e95cd194e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.863851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f747c9-8234-4d3f-b6f6-0e95cd194e24 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.889543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 807/1581] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.889543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f747c9-8234-4d3f-b6f6-0e95cd194e24 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.897690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f747c9-8234-4d3f-b6f6-0e95cd194e24 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:02.898284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979b80e-540c-4d07-b7ac-c02e62d52864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.898988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979b80e-540c-4d07-b7ac-c02e62d52864 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.899251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979b80e-540c-4d07-b7ac-c02e62d52864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.899564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979b80e-540c-4d07-b7ac-c02e62d52864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.900562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979b80e-540c-4d07-b7ac-c02e62d52864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.902533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979b80e-540c-4d07-b7ac-c02e62d52864 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.902959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:02.903618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:02.903897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:02.904170 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] PATH_INFO: `/v3/projects/c71fa2391435493e9533ca36af2426f2/users/d825945a618d4157b322ea0cd621af11/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:02.919224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c71fa2391435493e9533ca36af2426f2, user_id=d825945a618d4157b322ea0cd621af11, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:02.920293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:37:02.920293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 28 06:37:02.920293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: Jul 28 06:37:02.922359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:02.930893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 775/1582] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:02.931922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979b80e-540c-4d07-b7ac-c02e62d52864 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.936687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 808/1583] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:02 2022] PUT /identity/v3/projects/c71fa2391435493e9533ca36af2426f2/users/d825945a618d4157b322ea0cd621af11/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:02.938290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:02.947312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:02.973005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:03.003038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979b80e-540c-4d07-b7ac-c02e62d52864 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.008257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7affcf6-69e9-4acb-ba09-8fd018f79afd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.008565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb355c10-ad93-41ba-a1e5-1dc0b3569406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.009287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb355c10-ad93-41ba-a1e5-1dc0b3569406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.009608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb355c10-ad93-41ba-a1e5-1dc0b3569406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.009892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb355c10-ad93-41ba-a1e5-1dc0b3569406 admin admin] PATH_INFO: `/v3/projects/0fb2f4ad616e43f5a9a248390bd0aebf/users/a2080efc6cbd471995593012bc268765/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.014131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9904666-949f-48cc-9893-ab6bd8ac5ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.014835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904666-949f-48cc-9893-ab6bd8ac5ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.015211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904666-949f-48cc-9893-ab6bd8ac5ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.015581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904666-949f-48cc-9893-ab6bd8ac5ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.016339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9904666-949f-48cc-9893-ab6bd8ac5ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.018384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9904666-949f-48cc-9893-ab6bd8ac5ce6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.025099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb355c10-ad93-41ba-a1e5-1dc0b3569406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb2f4ad616e43f5a9a248390bd0aebf, user_id=a2080efc6cbd471995593012bc268765, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.027781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb355c10-ad93-41ba-a1e5-1dc0b3569406 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.041336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 776/1584] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:02 2022] PUT /identity/v3/projects/0fb2f4ad616e43f5a9a248390bd0aebf/users/a2080efc6cbd471995593012bc268765/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:03.042129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb355c10-ad93-41ba-a1e5-1dc0b3569406 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.050570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 809/1585] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:02 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.051506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9904666-949f-48cc-9893-ab6bd8ac5ce6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.104435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb355c10-ad93-41ba-a1e5-1dc0b3569406 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.115338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9904666-949f-48cc-9893-ab6bd8ac5ce6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.120124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295b7ff-e59f-4511-89f0-17c0245637cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.120774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295b7ff-e59f-4511-89f0-17c0245637cc admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.121060 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295b7ff-e59f-4511-89f0-17c0245637cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.121593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295b7ff-e59f-4511-89f0-17c0245637cc admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.122034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295b7ff-e59f-4511-89f0-17c0245637cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.124015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295b7ff-e59f-4511-89f0-17c0245637cc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.151770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 810/1586] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.151770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295b7ff-e59f-4511-89f0-17c0245637cc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.159001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295b7ff-e59f-4511-89f0-17c0245637cc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.164184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1bcfd-4044-4e51-9c15-247c22f57e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.164790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1bcfd-4044-4e51-9c15-247c22f57e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.165076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1bcfd-4044-4e51-9c15-247c22f57e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.165387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1bcfd-4044-4e51-9c15-247c22f57e52 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.165861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b29db-7738-416a-9958-40c61a8c608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.166044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1bcfd-4044-4e51-9c15-247c22f57e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.166485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b29db-7738-416a-9958-40c61a8c608f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.166770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b29db-7738-416a-9958-40c61a8c608f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.167049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b29db-7738-416a-9958-40c61a8c608f admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.167712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b29db-7738-416a-9958-40c61a8c608f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.167943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1bcfd-4044-4e51-9c15-247c22f57e52 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.169619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b29db-7738-416a-9958-40c61a8c608f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.192300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 811/1587] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.194102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1bcfd-4044-4e51-9c15-247c22f57e52 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.202878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 777/1588] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.203795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b29db-7738-416a-9958-40c61a8c608f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.203952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1bcfd-4044-4e51-9c15-247c22f57e52 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.233552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:03.263894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f7f60-6756-4070-a363-e15a0aa43bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.264515 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f7f60-6756-4070-a363-e15a0aa43bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.264800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f7f60-6756-4070-a363-e15a0aa43bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.265078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f7f60-6756-4070-a363-e15a0aa43bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.265810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f7f60-6756-4070-a363-e15a0aa43bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.266765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b29db-7738-416a-9958-40c61a8c608f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.267559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f7f60-6756-4070-a363-e15a0aa43bdd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.271710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e9a15-5954-49ce-9f85-19ed14e98f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.272323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9a15-5954-49ce-9f85-19ed14e98f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.272605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9a15-5954-49ce-9f85-19ed14e98f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.272877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9a15-5954-49ce-9f85-19ed14e98f64 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.273566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e9a15-5954-49ce-9f85-19ed14e98f64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.275455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e9a15-5954-49ce-9f85-19ed14e98f64 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.277854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 812/1589] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.279070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188db715-6dfa-4203-90a0-da7324a4a3d9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.279358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188db715-6dfa-4203-90a0-da7324a4a3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.279643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188db715-6dfa-4203-90a0-da7324a4a3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.298576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 778/1590] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.300562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f0046-5e7b-43bf-a679-03b64b1a2e03 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.301089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f0046-5e7b-43bf-a679-03b64b1a2e03 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.301621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f0046-5e7b-43bf-a679-03b64b1a2e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.311245 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-188db715-6dfa-4203-90a0-da7324a4a3d9 None None] MFA Rules not processed for user `d825945a618d4157b322ea0cd621af11`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:03.312494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-188db715-6dfa-4203-90a0-da7324a4a3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:03.322075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:03.323281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b94f0046-5e7b-43bf-a679-03b64b1a2e03 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:03.324531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b94f0046-5e7b-43bf-a679-03b64b1a2e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:03.332857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 779/1591] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:03.334420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f0046-5e7b-43bf-a679-03b64b1a2e03 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.345135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f0046-5e7b-43bf-a679-03b64b1a2e03 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.350073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b3691-cc78-417d-ac5f-379fbd9460bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.350684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3691-cc78-417d-ac5f-379fbd9460bf admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.350965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3691-cc78-417d-ac5f-379fbd9460bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.351237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3691-cc78-417d-ac5f-379fbd9460bf admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.351894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b3691-cc78-417d-ac5f-379fbd9460bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.353826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b3691-cc78-417d-ac5f-379fbd9460bf admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.377994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 780/1592] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.378871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b3691-cc78-417d-ac5f-379fbd9460bf admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.385208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 813/1593] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:03.385994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188db715-6dfa-4203-90a0-da7324a4a3d9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.391617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b3691-cc78-417d-ac5f-379fbd9460bf admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.396563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188db715-6dfa-4203-90a0-da7324a4a3d9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.397030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26d9bb-d73b-423f-912d-3f7542da94d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.397679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d9bb-d73b-423f-912d-3f7542da94d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.397959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d9bb-d73b-423f-912d-3f7542da94d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.398233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d9bb-d73b-423f-912d-3f7542da94d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.398920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26d9bb-d73b-423f-912d-3f7542da94d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.400550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26d9bb-d73b-423f-912d-3f7542da94d9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.401322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdc7ff-aa65-41db-a833-8cc0baa2cf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.401988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdc7ff-aa65-41db-a833-8cc0baa2cf9c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.402216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdc7ff-aa65-41db-a833-8cc0baa2cf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.402492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdc7ff-aa65-41db-a833-8cc0baa2cf9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.403185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdc7ff-aa65-41db-a833-8cc0baa2cf9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.404769 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdc7ff-aa65-41db-a833-8cc0baa2cf9c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.410645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 781/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.411546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26d9bb-d73b-423f-912d-3f7542da94d9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.418514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 814/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.418514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdc7ff-aa65-41db-a833-8cc0baa2cf9c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.424133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26d9bb-d73b-423f-912d-3f7542da94d9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.426779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdc7ff-aa65-41db-a833-8cc0baa2cf9c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.429563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.430209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.430550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.430777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.431457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.433402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.434887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587cea0-67a9-443f-bf26-5c107cb8f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.435582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587cea0-67a9-443f-bf26-5c107cb8f36d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.436003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587cea0-67a9-443f-bf26-5c107cb8f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.436334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587cea0-67a9-443f-bf26-5c107cb8f36d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.437131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587cea0-67a9-443f-bf26-5c107cb8f36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.438808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587cea0-67a9-443f-bf26-5c107cb8f36d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.452275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 815/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.456717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587cea0-67a9-443f-bf26-5c107cb8f36d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.457638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 782/1597] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/users => generated 291 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.458998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.460904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:03.469689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587cea0-67a9-443f-bf26-5c107cb8f36d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.475152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a50186-9f0f-45d8-b74b-c5231d240aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.475767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50186-9f0f-45d8-b74b-c5231d240aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.476053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50186-9f0f-45d8-b74b-c5231d240aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.476347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50186-9f0f-45d8-b74b-c5231d240aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.477029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a50186-9f0f-45d8-b74b-c5231d240aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.478670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a50186-9f0f-45d8-b74b-c5231d240aa6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.489649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 816/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.491233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a50186-9f0f-45d8-b74b-c5231d240aa6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.500594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a50186-9f0f-45d8-b74b-c5231d240aa6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.506694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0d69b-8b28-41c3-88db-4aca6a889c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.507303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0d69b-8b28-41c3-88db-4aca6a889c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.507591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0d69b-8b28-41c3-88db-4aca6a889c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.507913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0d69b-8b28-41c3-88db-4aca6a889c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.508605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0d69b-8b28-41c3-88db-4aca6a889c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.510248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0d69b-8b28-41c3-88db-4aca6a889c87 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.518788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279be007-778d-42be-b5ba-2ef0444e61be admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.524467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 817/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.524467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0d69b-8b28-41c3-88db-4aca6a889c87 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.524847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453c3e3-4c02-4afd-aba1-ab35461d29e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.524847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453c3e3-4c02-4afd-aba1-ab35461d29e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.524847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453c3e3-4c02-4afd-aba1-ab35461d29e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.524847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453c3e3-4c02-4afd-aba1-ab35461d29e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.531516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0d69b-8b28-41c3-88db-4aca6a889c87 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.533615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453c3e3-4c02-4afd-aba1-ab35461d29e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.535050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453c3e3-4c02-4afd-aba1-ab35461d29e9 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.536450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fade6-c602-414a-b123-8d132f6a407a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.537829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fade6-c602-414a-b123-8d132f6a407a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.538202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fade6-c602-414a-b123-8d132f6a407a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.538482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fade6-c602-414a-b123-8d132f6a407a admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.539139 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fade6-c602-414a-b123-8d132f6a407a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.541069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fade6-c602-414a-b123-8d132f6a407a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.545310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 783/1600] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:03.546280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453c3e3-4c02-4afd-aba1-ab35461d29e9 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.556126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453c3e3-4c02-4afd-aba1-ab35461d29e9 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.561967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaea2c-e699-43cd-89da-5dd8994f0800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.562992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaea2c-e699-43cd-89da-5dd8994f0800 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.563579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaea2c-e699-43cd-89da-5dd8994f0800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.564188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaea2c-e699-43cd-89da-5dd8994f0800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.565091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaea2c-e699-43cd-89da-5dd8994f0800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.566891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaea2c-e699-43cd-89da-5dd8994f0800 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.572458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 818/1601] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.573382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fade6-c602-414a-b123-8d132f6a407a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.581045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 784/1602] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.581916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaea2c-e699-43cd-89da-5dd8994f0800 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.594649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaea2c-e699-43cd-89da-5dd8994f0800 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.656228 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fade6-c602-414a-b123-8d132f6a407a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.657209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:03.664782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa4d7-8b4b-4a4f-bcc8-929180255271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.665742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa4d7-8b4b-4a4f-bcc8-929180255271 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.666317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa4d7-8b4b-4a4f-bcc8-929180255271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.666752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa4d7-8b4b-4a4f-bcc8-929180255271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.666882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57bc1f-7fe9-44af-82b7-ecdbe10d49a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.667611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57bc1f-7fe9-44af-82b7-ecdbe10d49a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.668172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fa4d7-8b4b-4a4f-bcc8-929180255271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.668302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57bc1f-7fe9-44af-82b7-ecdbe10d49a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.668504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57bc1f-7fe9-44af-82b7-ecdbe10d49a2 admin admin] PATH_INFO: `/v3/projects/d9e0dc09f9844b57bf7b38a5d4e54e17/users/cf13ae714ad04d6696c273603bba9fbc/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.671255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fa4d7-8b4b-4a4f-bcc8-929180255271 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.683950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 819/1603] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.684962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa4d7-8b4b-4a4f-bcc8-929180255271 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.687210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57bc1f-7fe9-44af-82b7-ecdbe10d49a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e0dc09f9844b57bf7b38a5d4e54e17, user_id=cf13ae714ad04d6696c273603bba9fbc, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.691400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57bc1f-7fe9-44af-82b7-ecdbe10d49a2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.696952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa4d7-8b4b-4a4f-bcc8-929180255271 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.702723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a97c2-4c5e-4415-af2a-91bdedfd788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.703481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a97c2-4c5e-4415-af2a-91bdedfd788b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.703826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a97c2-4c5e-4415-af2a-91bdedfd788b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.704166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a97c2-4c5e-4415-af2a-91bdedfd788b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.704970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a97c2-4c5e-4415-af2a-91bdedfd788b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.706700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a97c2-4c5e-4415-af2a-91bdedfd788b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.710501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 785/1604] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:03 2022] PUT /identity/v3/projects/d9e0dc09f9844b57bf7b38a5d4e54e17/users/cf13ae714ad04d6696c273603bba9fbc/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:03.711709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57bc1f-7fe9-44af-82b7-ecdbe10d49a2 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.719872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 820/1605] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.720734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a97c2-4c5e-4415-af2a-91bdedfd788b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.731207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a97c2-4c5e-4415-af2a-91bdedfd788b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.763660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:03.802956 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57bc1f-7fe9-44af-82b7-ecdbe10d49a2 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.816325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403babc-2e40-4a7e-93e2-930368f5d545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.816555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d66f0a-e5af-417e-bd4f-fe09eb59a3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.817329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d66f0a-e5af-417e-bd4f-fe09eb59a3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.817733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403babc-2e40-4a7e-93e2-930368f5d545 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.818459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d66f0a-e5af-417e-bd4f-fe09eb59a3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.818720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403babc-2e40-4a7e-93e2-930368f5d545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.818909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d66f0a-e5af-417e-bd4f-fe09eb59a3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.819286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403babc-2e40-4a7e-93e2-930368f5d545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.819837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d66f0a-e5af-417e-bd4f-fe09eb59a3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.820499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403babc-2e40-4a7e-93e2-930368f5d545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.821729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d66f0a-e5af-417e-bd4f-fe09eb59a3b2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.822469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403babc-2e40-4a7e-93e2-930368f5d545 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.835058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 786/1606] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.838731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 821/1607] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:03.839267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403babc-2e40-4a7e-93e2-930368f5d545 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.840845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d66f0a-e5af-417e-bd4f-fe09eb59a3b2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.847075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403babc-2e40-4a7e-93e2-930368f5d545 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.852028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae0e32-0b07-41a1-8b08-bc42b1223eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.852651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae0e32-0b07-41a1-8b08-bc42b1223eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.852939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae0e32-0b07-41a1-8b08-bc42b1223eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.853122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d66f0a-e5af-417e-bd4f-fe09eb59a3b2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.853241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae0e32-0b07-41a1-8b08-bc42b1223eb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.853910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae0e32-0b07-41a1-8b08-bc42b1223eb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.856435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae0e32-0b07-41a1-8b08-bc42b1223eb4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.869978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33743b94-47db-4278-9823-cf0ba1d296d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.874739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743b94-47db-4278-9823-cf0ba1d296d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.875441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743b94-47db-4278-9823-cf0ba1d296d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.876030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743b94-47db-4278-9823-cf0ba1d296d6 admin admin] PATH_INFO: `/v3/projects/dcffc8bbe6ec425997d67634a0f7d724/users/4a1db92e28444854b9de817dd36104ea/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.884683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 787/1608] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/users => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:03.885792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae0e32-0b07-41a1-8b08-bc42b1223eb4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:03.895582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33743b94-47db-4278-9823-cf0ba1d296d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcffc8bbe6ec425997d67634a0f7d724, user_id=4a1db92e28444854b9de817dd36104ea, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.899395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33743b94-47db-4278-9823-cf0ba1d296d6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.900434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae0e32-0b07-41a1-8b08-bc42b1223eb4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:03.908924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3a382-f377-4bbb-93e1-b7a84e4464fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:03.908924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3a382-f377-4bbb-93e1-b7a84e4464fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.908924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3a382-f377-4bbb-93e1-b7a84e4464fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.908924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3a382-f377-4bbb-93e1-b7a84e4464fe admin admin] PATH_INFO: `/v3/projects/a13254920b734a24948b0bdcad37f37b/users/6ba9ce6cb5404b358a3c9591a7aa1775/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.928344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3a382-f377-4bbb-93e1-b7a84e4464fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13254920b734a24948b0bdcad37f37b, user_id=6ba9ce6cb5404b358a3c9591a7aa1775, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:03.931071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3a382-f377-4bbb-93e1-b7a84e4464fe admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:03.932044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 822/1609] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:03 2022] PUT /identity/v3/projects/dcffc8bbe6ec425997d67634a0f7d724/users/4a1db92e28444854b9de817dd36104ea/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:03.933754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092279-077c-4d37-be09-d013e438fd5e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:03.933912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092279-077c-4d37-be09-d013e438fd5e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:03.934191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc092279-077c-4d37-be09-d013e438fd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:03.951290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 788/1610] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:03 2022] PUT /identity/v3/projects/a13254920b734a24948b0bdcad37f37b/users/6ba9ce6cb5404b358a3c9591a7aa1775/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:03.957859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cc092279-077c-4d37-be09-d013e438fd5e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:03.960181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cc092279-077c-4d37-be09-d013e438fd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:03.966110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3a382-f377-4bbb-93e1-b7a84e4464fe admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.042392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 823/1611] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:04.043684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc092279-077c-4d37-be09-d013e438fd5e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.055812 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3a382-f377-4bbb-93e1-b7a84e4464fe admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.057534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc092279-077c-4d37-be09-d013e438fd5e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.061194 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efccd1-3c68-4787-809f-6e964ef0a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.061943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efccd1-3c68-4787-809f-6e964ef0a944 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.062083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70795f-51a9-4bdf-a9b8-b41bdb5f8913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.062256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efccd1-3c68-4787-809f-6e964ef0a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.062632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efccd1-3c68-4787-809f-6e964ef0a944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.063347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70795f-51a9-4bdf-a9b8-b41bdb5f8913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.063456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efccd1-3c68-4787-809f-6e964ef0a944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.063662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70795f-51a9-4bdf-a9b8-b41bdb5f8913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.063912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70795f-51a9-4bdf-a9b8-b41bdb5f8913 admin admin] PATH_INFO: `/v3/projects/a0e7a5dae78a4c10926c6fb66458c71b/users/b240f68acc734bd492843ad4ac35807c/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.065251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efccd1-3c68-4787-809f-6e964ef0a944 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.082616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 789/1612] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:04.084051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efccd1-3c68-4787-809f-6e964ef0a944 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.087957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70795f-51a9-4bdf-a9b8-b41bdb5f8913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e7a5dae78a4c10926c6fb66458c71b, user_id=b240f68acc734bd492843ad4ac35807c, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.090710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70795f-51a9-4bdf-a9b8-b41bdb5f8913 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.096387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efccd1-3c68-4787-809f-6e964ef0a944 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.105406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603c6d1-b9d5-435b-a80d-051390ac6b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.106175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603c6d1-b9d5-435b-a80d-051390ac6b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.106766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603c6d1-b9d5-435b-a80d-051390ac6b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.106894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 824/1613] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:04 2022] PUT /identity/v3/projects/a0e7a5dae78a4c10926c6fb66458c71b/users/b240f68acc734bd492843ad4ac35807c/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:04.107667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70795f-51a9-4bdf-a9b8-b41bdb5f8913 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.110468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603c6d1-b9d5-435b-a80d-051390ac6b93 admin admin] PATH_INFO: `/v3/projects/b9fa6550ed7d4062bbd3e8083e7bdc22/users/1c3e82a7c47d4d4499fab08b2cf30854/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.126531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603c6d1-b9d5-435b-a80d-051390ac6b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fa6550ed7d4062bbd3e8083e7bdc22, user_id=1c3e82a7c47d4d4499fab08b2cf30854, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.129890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603c6d1-b9d5-435b-a80d-051390ac6b93 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.144881 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 790/1614] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:04 2022] PUT /identity/v3/projects/b9fa6550ed7d4062bbd3e8083e7bdc22/users/1c3e82a7c47d4d4499fab08b2cf30854/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:04.147076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7875a6-7e46-4629-9a9d-d2d8fbd370a0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.147583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7875a6-7e46-4629-9a9d-d2d8fbd370a0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.147866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7875a6-7e46-4629-9a9d-d2d8fbd370a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.156963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:04.173630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ba7875a6-7e46-4629-9a9d-d2d8fbd370a0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:04.175031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ba7875a6-7e46-4629-9a9d-d2d8fbd370a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:04.196759 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70795f-51a9-4bdf-a9b8-b41bdb5f8913 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.229533 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:04.247708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 791/1615] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:04.249046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e25e94-fdf4-45bd-8374-5fbdab46ba75 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.249250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e25e94-fdf4-45bd-8374-5fbdab46ba75 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.249664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e25e94-fdf4-45bd-8374-5fbdab46ba75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.278703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4475f-97c4-45f2-9a52-2badf3e2028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.279398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4475f-97c4-45f2-9a52-2badf3e2028b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.279686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4475f-97c4-45f2-9a52-2badf3e2028b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.279988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4475f-97c4-45f2-9a52-2badf3e2028b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.280689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4475f-97c4-45f2-9a52-2badf3e2028b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.282362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4475f-97c4-45f2-9a52-2badf3e2028b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.288648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a0e25e94-fdf4-45bd-8374-5fbdab46ba75 None None] MFA Rules not processed for user `6ba9ce6cb5404b358a3c9591a7aa1775`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:04.289921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a0e25e94-fdf4-45bd-8374-5fbdab46ba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:04.294798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 825/1616] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:04.295597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4475f-97c4-45f2-9a52-2badf3e2028b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.359293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4475f-97c4-45f2-9a52-2badf3e2028b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.364159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6104ba-2b42-4478-81f9-d0eef434cec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.366429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6104ba-2b42-4478-81f9-d0eef434cec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.366829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6104ba-2b42-4478-81f9-d0eef434cec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.372626 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6104ba-2b42-4478-81f9-d0eef434cec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.386312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 792/1617] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:04.387420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e25e94-fdf4-45bd-8374-5fbdab46ba75 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.395608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6104ba-2b42-4478-81f9-d0eef434cec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.399404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6104ba-2b42-4478-81f9-d0eef434cec8 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.401813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e25e94-fdf4-45bd-8374-5fbdab46ba75 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.407806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9774f3-a4a7-4d79-9c9f-db5b5139b6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.408549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9774f3-a4a7-4d79-9c9f-db5b5139b6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.408837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9774f3-a4a7-4d79-9c9f-db5b5139b6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.409121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9774f3-a4a7-4d79-9c9f-db5b5139b6cf admin admin] PATH_INFO: `/v3/projects/1b2913c373d244a2987135e3dc4cdac4/users/b51b943f169742f28a9e6222bf04c986/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.414298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 826/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:04.415305 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36ff9e-864e-463d-b21e-c1cf05283a70 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.415808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36ff9e-864e-463d-b21e-c1cf05283a70 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.416172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36ff9e-864e-463d-b21e-c1cf05283a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.430204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9774f3-a4a7-4d79-9c9f-db5b5139b6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b2913c373d244a2987135e3dc4cdac4, user_id=b51b943f169742f28a9e6222bf04c986, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.434239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9774f3-a4a7-4d79-9c9f-db5b5139b6cf admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.442787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0a36ff9e-864e-463d-b21e-c1cf05283a70 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:04.444260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0a36ff9e-864e-463d-b21e-c1cf05283a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:04.452501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 793/1619] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:04 2022] PUT /identity/v3/projects/1b2913c373d244a2987135e3dc4cdac4/users/b51b943f169742f28a9e6222bf04c986/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:04.464841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa726-68d0-40bb-be34-50ecea3b3a0b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.466387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa726-68d0-40bb-be34-50ecea3b3a0b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.467814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa726-68d0-40bb-be34-50ecea3b3a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.515960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1f0aa726-68d0-40bb-be34-50ecea3b3a0b None None] MFA Rules not processed for user `1c3e82a7c47d4d4499fab08b2cf30854`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:04.517664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1f0aa726-68d0-40bb-be34-50ecea3b3a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:04.561344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 827/1620] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:04.561344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36ff9e-864e-463d-b21e-c1cf05283a70 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.600066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:04.605967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 794/1621] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:04.608651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aa726-68d0-40bb-be34-50ecea3b3a0b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.628566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aa726-68d0-40bb-be34-50ecea3b3a0b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.637003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb704f0-14e3-4ac5-8de6-ceaa928112c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.637997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb704f0-14e3-4ac5-8de6-ceaa928112c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.638696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb704f0-14e3-4ac5-8de6-ceaa928112c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.639040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb704f0-14e3-4ac5-8de6-ceaa928112c7 admin admin] PATH_INFO: `/v3/projects/009965e800bc4eef8def39a5cd382ab4/users/4a902783d9f34ed9988958e90e012ab9/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.645145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36ff9e-864e-463d-b21e-c1cf05283a70 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.651077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0da70-76d5-4b37-b30e-ef279f56be16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.651849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0da70-76d5-4b37-b30e-ef279f56be16 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.652285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0da70-76d5-4b37-b30e-ef279f56be16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.652635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0da70-76d5-4b37-b30e-ef279f56be16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.661034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb704f0-14e3-4ac5-8de6-ceaa928112c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009965e800bc4eef8def39a5cd382ab4, user_id=4a902783d9f34ed9988958e90e012ab9, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.666252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0da70-76d5-4b37-b30e-ef279f56be16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.666413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb704f0-14e3-4ac5-8de6-ceaa928112c7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.667671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0da70-76d5-4b37-b30e-ef279f56be16 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.682192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 828/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:04.684007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6e1f9-3db4-412b-8c1d-7a2063098ad1 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.684668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6e1f9-3db4-412b-8c1d-7a2063098ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.685092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6e1f9-3db4-412b-8c1d-7a2063098ad1 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.689586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 795/1623] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:04 2022] PUT /identity/v3/projects/009965e800bc4eef8def39a5cd382ab4/users/4a902783d9f34ed9988958e90e012ab9/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:04.689586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709c58a-c89f-49a6-b9bd-4a555a102c61 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.689586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709c58a-c89f-49a6-b9bd-4a555a102c61 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.689929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709c58a-c89f-49a6-b9bd-4a555a102c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.692612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 829/1624] 10.222.0.20 () {54 vars in 962 bytes} [Thu Jul 28 06:37:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:37:04.693606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6e1f9-3db4-412b-8c1d-7a2063098ad1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.707791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:04.741327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e709c58a-c89f-49a6-b9bd-4a555a102c61 None None] MFA Rules not processed for user `b51b943f169742f28a9e6222bf04c986`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:04.743209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e709c58a-c89f-49a6-b9bd-4a555a102c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:04.777141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6e1f9-3db4-412b-8c1d-7a2063098ad1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.783650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa164-52b9-40ae-a970-ada5f6f280d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.784611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa164-52b9-40ae-a970-ada5f6f280d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.784968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa164-52b9-40ae-a970-ada5f6f280d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.785739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa164-52b9-40ae-a970-ada5f6f280d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.852677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 796/1625] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:04.854165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709c58a-c89f-49a6-b9bd-4a555a102c61 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.868832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aa164-52b9-40ae-a970-ada5f6f280d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.870906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aa164-52b9-40ae-a970-ada5f6f280d0 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.897464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 830/1626] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:04.898767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa164-52b9-40ae-a970-ada5f6f280d0 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:04.924209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa164-52b9-40ae-a970-ada5f6f280d0 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.937114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab33ac9-73c4-4c2c-9d60-4750209a21f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.937880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab33ac9-73c4-4c2c-9d60-4750209a21f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.938351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab33ac9-73c4-4c2c-9d60-4750209a21f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.938643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab33ac9-73c4-4c2c-9d60-4750209a21f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.939495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab33ac9-73c4-4c2c-9d60-4750209a21f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.941993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab33ac9-73c4-4c2c-9d60-4750209a21f0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:04.975401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709c58a-c89f-49a6-b9bd-4a555a102c61 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:04.980177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 831/1627] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:04.980177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2783ec-6706-48b0-ab4c-6171ce0b4a18 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.980177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2783ec-6706-48b0-ab4c-6171ce0b4a18 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.980177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2783ec-6706-48b0-ab4c-6171ce0b4a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.982510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f9cc9-14ca-41ce-84db-70da3b23e558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:04.983530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f9cc9-14ca-41ce-84db-70da3b23e558 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:04.983917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f9cc9-14ca-41ce-84db-70da3b23e558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:04.984304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f9cc9-14ca-41ce-84db-70da3b23e558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:04.985080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f9cc9-14ca-41ce-84db-70da3b23e558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:04.987721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f9cc9-14ca-41ce-84db-70da3b23e558 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.044417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 797/1628] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:04 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:05.045454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f9cc9-14ca-41ce-84db-70da3b23e558 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.048965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bf2783ec-6706-48b0-ab4c-6171ce0b4a18 None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:05.050331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bf2783ec-6706-48b0-ab4c-6171ce0b4a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:05.140617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 832/1629] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Jul 28 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:05.140971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2783ec-6706-48b0-ab4c-6171ce0b4a18 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.171119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:05.185075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:05.195863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f9cc9-14ca-41ce-84db-70da3b23e558 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.203720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7efe2f-6b25-428c-ba45-59ec4a22994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.204694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7efe2f-6b25-428c-ba45-59ec4a22994b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.205271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7efe2f-6b25-428c-ba45-59ec4a22994b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.205765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7efe2f-6b25-428c-ba45-59ec4a22994b admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.206848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7efe2f-6b25-428c-ba45-59ec4a22994b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.210943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7efe2f-6b25-428c-ba45-59ec4a22994b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.269127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 798/1630] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:05.270404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7efe2f-6b25-428c-ba45-59ec4a22994b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.290191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2783ec-6706-48b0-ab4c-6171ce0b4a18 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.366241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7efe2f-6b25-428c-ba45-59ec4a22994b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.373274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ed73-e67b-4296-9419-8f3666ab95f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.374357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ed73-e67b-4296-9419-8f3666ab95f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.374907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ed73-e67b-4296-9419-8f3666ab95f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.375396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ed73-e67b-4296-9419-8f3666ab95f1 admin admin] PATH_INFO: `/v3/projects/009965e800bc4eef8def39a5cd382ab4/users/4a902783d9f34ed9988958e90e012ab9/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.376876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb213a0f-8da0-4821-82e4-aef16880ec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.377535 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213a0f-8da0-4821-82e4-aef16880ec42 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.377815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213a0f-8da0-4821-82e4-aef16880ec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.378223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213a0f-8da0-4821-82e4-aef16880ec42 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.378903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb213a0f-8da0-4821-82e4-aef16880ec42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.379475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976ed73-e67b-4296-9419-8f3666ab95f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009965e800bc4eef8def39a5cd382ab4, user_id=4a902783d9f34ed9988958e90e012ab9, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.382078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb213a0f-8da0-4821-82e4-aef16880ec42 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.384163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976ed73-e67b-4296-9419-8f3666ab95f1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.398298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 833/1631] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:05 2022] PUT /identity/v3/projects/009965e800bc4eef8def39a5cd382ab4/users/4a902783d9f34ed9988958e90e012ab9/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:05.407132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ed73-e67b-4296-9419-8f3666ab95f1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.422199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 799/1632] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:05.423326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb213a0f-8da0-4821-82e4-aef16880ec42 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.465625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb213a0f-8da0-4821-82e4-aef16880ec42 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.520733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ed73-e67b-4296-9419-8f3666ab95f1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.533247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7a6c1-1058-48dc-9913-b898bade58b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.534196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7a6c1-1058-48dc-9913-b898bade58b4 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.534573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7a6c1-1058-48dc-9913-b898bade58b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.534986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7a6c1-1058-48dc-9913-b898bade58b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.595715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa875e9f-9000-416a-b643-b5bfa637d6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.596456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa875e9f-9000-416a-b643-b5bfa637d6bc admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.596942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa875e9f-9000-416a-b643-b5bfa637d6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.597362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa875e9f-9000-416a-b643-b5bfa637d6bc admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.598125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa875e9f-9000-416a-b643-b5bfa637d6bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.602170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa875e9f-9000-416a-b643-b5bfa637d6bc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.639842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7a6c1-1058-48dc-9913-b898bade58b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.650066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 800/1633] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:05.650066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4fc6e-68d8-441e-9c0d-eb57f355f881 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.650066 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4fc6e-68d8-441e-9c0d-eb57f355f881 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.650440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4fc6e-68d8-441e-9c0d-eb57f355f881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.654644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7a6c1-1058-48dc-9913-b898bade58b4 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.672756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 834/1634] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:05.686521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1de4fc6e-68d8-441e-9c0d-eb57f355f881 None None] MFA Rules not processed for user `4a902783d9f34ed9988958e90e012ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:05.688006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7a6c1-1058-48dc-9913-b898bade58b4 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.688690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1de4fc6e-68d8-441e-9c0d-eb57f355f881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:05.719279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:05.821519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7a6c1-1058-48dc-9913-b898bade58b4 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.821519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e64550-b128-4dde-b172-91e11453d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.824568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:05.824844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e64550-b128-4dde-b172-91e11453d649 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.824844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e64550-b128-4dde-b172-91e11453d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.824844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e64550-b128-4dde-b172-91e11453d649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.824844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e64550-b128-4dde-b172-91e11453d649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.824844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e64550-b128-4dde-b172-91e11453d649 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.836258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 801/1635] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:05.836258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4fc6e-68d8-441e-9c0d-eb57f355f881 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.854456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 835/1636] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:05.855924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e64550-b128-4dde-b172-91e11453d649 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.858290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4fc6e-68d8-441e-9c0d-eb57f355f881 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.882980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe56b7-41a9-429f-a2e0-f08ae6a035ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.883870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe56b7-41a9-429f-a2e0-f08ae6a035ad admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.884262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe56b7-41a9-429f-a2e0-f08ae6a035ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.884647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe56b7-41a9-429f-a2e0-f08ae6a035ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.885675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe56b7-41a9-429f-a2e0-f08ae6a035ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:05.888433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe56b7-41a9-429f-a2e0-f08ae6a035ad admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:05.916382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 802/1637] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:05.929457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe56b7-41a9-429f-a2e0-f08ae6a035ad admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:05.953624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe56b7-41a9-429f-a2e0-f08ae6a035ad admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:05.961095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a725a-9f05-457d-ac78-8204ff072940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:05.962660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a725a-9f05-457d-ac78-8204ff072940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:05.962660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a725a-9f05-457d-ac78-8204ff072940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:05.962660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a725a-9f05-457d-ac78-8204ff072940 admin admin] PATH_INFO: `/v3/projects/acc8de87dc104acbae3e42246993aaf5/users/667acf048de749ea8988da92b835c240/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:05.998496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a725a-9f05-457d-ac78-8204ff072940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc8de87dc104acbae3e42246993aaf5, user_id=667acf048de749ea8988da92b835c240, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.001305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a725a-9f05-457d-ac78-8204ff072940 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.015931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e64550-b128-4dde-b172-91e11453d649 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.031325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 803/1638] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:05 2022] PUT /identity/v3/projects/acc8de87dc104acbae3e42246993aaf5/users/667acf048de749ea8988da92b835c240/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:06.032309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a725a-9f05-457d-ac78-8204ff072940 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.033262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b482d-b78b-428a-8d69-824fbca0c373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.034033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b482d-b78b-428a-8d69-824fbca0c373 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.035614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b482d-b78b-428a-8d69-824fbca0c373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.038210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b482d-b78b-428a-8d69-824fbca0c373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.066262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b482d-b78b-428a-8d69-824fbca0c373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.070553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b482d-b78b-428a-8d69-824fbca0c373 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.081899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 836/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:06.093348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b482d-b78b-428a-8d69-824fbca0c373 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.117242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b482d-b78b-428a-8d69-824fbca0c373 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.151411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a725a-9f05-457d-ac78-8204ff072940 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.158878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97d098-62a2-4972-8294-8e37ec54a4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.159768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97d098-62a2-4972-8294-8e37ec54a4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.160141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97d098-62a2-4972-8294-8e37ec54a4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.160551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97d098-62a2-4972-8294-8e37ec54a4f9 admin admin] PATH_INFO: `/v3/projects/92489cc2be1a4816b5abace67a136928/users/9c15d942187c4f6fb7cf62df543949c8/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.184359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97d098-62a2-4972-8294-8e37ec54a4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92489cc2be1a4816b5abace67a136928, user_id=9c15d942187c4f6fb7cf62df543949c8, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.187772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97d098-62a2-4972-8294-8e37ec54a4f9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.212159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 804/1640] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:06 2022] PUT /identity/v3/projects/92489cc2be1a4816b5abace67a136928/users/9c15d942187c4f6fb7cf62df543949c8/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:06.229046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97d098-62a2-4972-8294-8e37ec54a4f9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.237958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a6625-e0d1-454c-9796-8c9872e9d6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.238849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a6625-e0d1-454c-9796-8c9872e9d6cf admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.239287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a6625-e0d1-454c-9796-8c9872e9d6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.239819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a6625-e0d1-454c-9796-8c9872e9d6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.240938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a6625-e0d1-454c-9796-8c9872e9d6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.242765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a6625-e0d1-454c-9796-8c9872e9d6cf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.254753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 837/1641] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:06.258803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a6625-e0d1-454c-9796-8c9872e9d6cf admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.276888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a6625-e0d1-454c-9796-8c9872e9d6cf admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.347012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97d098-62a2-4972-8294-8e37ec54a4f9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.372079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699284f0-79cf-4df1-baea-aaeeb5d91c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.372893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699284f0-79cf-4df1-baea-aaeeb5d91c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.373318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699284f0-79cf-4df1-baea-aaeeb5d91c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.373705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699284f0-79cf-4df1-baea-aaeeb5d91c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.375383 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699284f0-79cf-4df1-baea-aaeeb5d91c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.383836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699284f0-79cf-4df1-baea-aaeeb5d91c3b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.425142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 805/1642] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:06.426646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b7d1c-103b-4161-9574-3134d61e5183 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.426942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b7d1c-103b-4161-9574-3134d61e5183 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.427761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b7d1c-103b-4161-9574-3134d61e5183 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.445093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 806/1643] 10.222.0.20 () {54 vars in 912 bytes} [Thu Jul 28 06:37:06 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:37:06.446433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b7d1c-103b-4161-9574-3134d61e5183 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:06.448127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b50b8e-b17b-4269-8d3e-fa961c344b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.448870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b50b8e-b17b-4269-8d3e-fa961c344b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.449159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b50b8e-b17b-4269-8d3e-fa961c344b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.449559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b50b8e-b17b-4269-8d3e-fa961c344b4d admin admin] PATH_INFO: `/v3/projects/acc8de87dc104acbae3e42246993aaf5/users/667acf048de749ea8988da92b835c240/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.465912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b50b8e-b17b-4269-8d3e-fa961c344b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc8de87dc104acbae3e42246993aaf5, user_id=667acf048de749ea8988da92b835c240, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.465912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b50b8e-b17b-4269-8d3e-fa961c344b4d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.482059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b7d1c-103b-4161-9574-3134d61e5183 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:06.494917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 838/1644] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:06 2022] PUT /identity/v3/projects/acc8de87dc104acbae3e42246993aaf5/users/667acf048de749ea8988da92b835c240/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:06.496405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54495cb-24da-4612-bec2-9ea62c9d77e8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.496701 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54495cb-24da-4612-bec2-9ea62c9d77e8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.496984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54495cb-24da-4612-bec2-9ea62c9d77e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.534011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:06.538854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d54495cb-24da-4612-bec2-9ea62c9d77e8 None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:06.543229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d54495cb-24da-4612-bec2-9ea62c9d77e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:06.591184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:06.660313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d39aeb-b0e0-4605-a956-ba88acf4b2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:06.661023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d39aeb-b0e0-4605-a956-ba88acf4b2f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.661325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d39aeb-b0e0-4605-a956-ba88acf4b2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.662192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d39aeb-b0e0-4605-a956-ba88acf4b2f6 admin admin] PATH_INFO: `/v3/projects/92489cc2be1a4816b5abace67a136928/users/9c15d942187c4f6fb7cf62df543949c8/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.672515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d39aeb-b0e0-4605-a956-ba88acf4b2f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92489cc2be1a4816b5abace67a136928, user_id=9c15d942187c4f6fb7cf62df543949c8, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:06.675584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d39aeb-b0e0-4605-a956-ba88acf4b2f6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:06.714402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 807/1645] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:06 2022] PUT /identity/v3/projects/92489cc2be1a4816b5abace67a136928/users/9c15d942187c4f6fb7cf62df543949c8/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:06.715912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011bba6-9c62-437d-8a3d-89f4819b8e2d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.716201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011bba6-9c62-437d-8a3d-89f4819b8e2d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.716483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011bba6-9c62-437d-8a3d-89f4819b8e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.725224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 839/1646] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:37:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:06.726282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cd80-5147-499e-aab7-1043868c0852 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:06.726575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cd80-5147-499e-aab7-1043868c0852 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:06.726853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cd80-5147-499e-aab7-1043868c0852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:06.815616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f011bba6-9c62-437d-8a3d-89f4819b8e2d None None] MFA Rules not processed for user `667acf048de749ea8988da92b835c240`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:06.817129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f011bba6-9c62-437d-8a3d-89f4819b8e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:06.831942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bbb5cd80-5147-499e-aab7-1043868c0852 None None] MFA Rules not processed for user `9c15d942187c4f6fb7cf62df543949c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:06.833237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bbb5cd80-5147-499e-aab7-1043868c0852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:06.985037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 808/1647] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:06.985956 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011bba6-9c62-437d-8a3d-89f4819b8e2d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.020595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 840/1648] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:07.022192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5cd80-5147-499e-aab7-1043868c0852 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.102700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5cd80-5147-499e-aab7-1043868c0852 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.110115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.111008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.111390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.111776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] PATH_INFO: `/v3/users/d825945a618d4157b322ea0cd621af11` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.112969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d825945a618d4157b322ea0cd621af11)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.114202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:37:07.114202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: warnings.warn(msg) Jul 28 06:37:07.114202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: Jul 28 06:37:07.115851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.134386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011bba6-9c62-437d-8a3d-89f4819b8e2d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.147910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd6232-a98c-4ba6-9df3-5ff008e7c276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.148736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6232-a98c-4ba6-9df3-5ff008e7c276 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.149027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6232-a98c-4ba6-9df3-5ff008e7c276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.149331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6232-a98c-4ba6-9df3-5ff008e7c276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.216250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:07.224945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd825945a618d4157b322ea0cd621af11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.238433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd825945a618d4157b322ea0cd621af11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.252192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd825945a618d4157b322ea0cd621af11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.252945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd825945a618d4157b322ea0cd621af11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.278480 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:07.303038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd825945a618d4157b322ea0cd621af11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:07.311981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd6232-a98c-4ba6-9df3-5ff008e7c276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.314296 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd6232-a98c-4ba6-9df3-5ff008e7c276 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.320449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 841/1649] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:07 2022] DELETE /identity/v3/users/d825945a618d4157b322ea0cd621af11 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:07.320449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.322568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.357852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 809/1650] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:07.361716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb3e88-0db2-425d-8b93-bbbb511ae356 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.364117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb3e88-0db2-425d-8b93-bbbb511ae356 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.365387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb3e88-0db2-425d-8b93-bbbb511ae356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.426657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75475a3f-4f62-40c2-aa51-e3909f7f89e9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.436463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.437117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.437444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.437776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.439676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.447355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6afb3e88-0db2-425d-8b93-bbbb511ae356 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:07.452007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6afb3e88-0db2-425d-8b93-bbbb511ae356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.623532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.627552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.633599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 810/1651] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:07.662806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 842/1652] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:07.706487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb3e88-0db2-425d-8b93-bbbb511ae356 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.707921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6afb3e88-0db2-425d-8b93-bbbb511ae356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.777192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:07.789714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f1ce-5ac2-414f-8951-e97845f1c412 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.805522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.808229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.808785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.809083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.815358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:07.838946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb3e88-0db2-425d-8b93-bbbb511ae356 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:07.844356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d74eb4-c427-4406-9576-a314c7eeb3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:07.845545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d74eb4-c427-4406-9576-a314c7eeb3ab service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:07.846192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d74eb4-c427-4406-9576-a314c7eeb3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:07.846730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d74eb4-c427-4406-9576-a314c7eeb3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:07.863258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d74eb4-c427-4406-9576-a314c7eeb3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:07.867891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d74eb4-c427-4406-9576-a314c7eeb3ab service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:07.885135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 811/1653] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:08.019784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:08.023930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:08.039516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 843/1654] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:08.271175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d74eb4-c427-4406-9576-a314c7eeb3ab service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:08.293780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d74eb4-c427-4406-9576-a314c7eeb3ab service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:08.310268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:08.310951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:08.311237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:08.311516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:08.313569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:08.456425 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:08.458163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:08.473400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 812/1655] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:09.127384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.134712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:09.139917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace89f7-b3f4-404c-9cfb-608b779d9baa service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.156004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cc3d1-55b0-4944-9dfb-4c3474bd4980 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c15d942187c4f6fb7cf62df543949c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92489cc2be1a4816b5abace67a136928', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.156888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cc3d1-55b0-4944-9dfb-4c3474bd4980 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.165655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cc3d1-55b0-4944-9dfb-4c3474bd4980 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.165944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cc3d1-55b0-4944-9dfb-4c3474bd4980 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.166696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cc3d1-55b0-4944-9dfb-4c3474bd4980 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:09.167007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cc3d1-55b0-4944-9dfb-4c3474bd4980 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:09.168637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cc3d1-55b0-4944-9dfb-4c3474bd4980 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:09.193092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 844/1656] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:09.198584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.214256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d044f-1e96-4cf6-b447-1d99fe5dcee5 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.221692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7491f7-9ea1-4f31-a79c-c0c022c65e41 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c15d942187c4f6fb7cf62df543949c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92489cc2be1a4816b5abace67a136928', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.222380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7491f7-9ea1-4f31-a79c-c0c022c65e41 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.222671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7491f7-9ea1-4f31-a79c-c0c022c65e41 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.222947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7491f7-9ea1-4f31-a79c-c0c022c65e41 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.225946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7491f7-9ea1-4f31-a79c-c0c022c65e41 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:09.228027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7491f7-9ea1-4f31-a79c-c0c022c65e41 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:09.277080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ba897-d1de-480a-8849-6fded0df60ce None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.277455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ba897-d1de-480a-8849-6fded0df60ce None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.277661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ba897-d1de-480a-8849-6fded0df60ce None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.279028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 845/1657] 10.222.0.20 () {54 vars in 962 bytes} [Thu Jul 28 06:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:37:09.282604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 813/1658] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:09 2022] POST /identity/v3/projects => generated 353 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:09.287783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7787d48-ecb3-4818-b25a-f38e2b089ae8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.288070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7787d48-ecb3-4818-b25a-f38e2b089ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.288346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7787d48-ecb3-4818-b25a-f38e2b089ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.289483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7491f7-9ea1-4f31-a79c-c0c022c65e41 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.297049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:09.312349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a7787d48-ecb3-4818-b25a-f38e2b089ae8 None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:09.313612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a7787d48-ecb3-4818-b25a-f38e2b089ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:09.386210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7491f7-9ea1-4f31-a79c-c0c022c65e41 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.395537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c15d942187c4f6fb7cf62df543949c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92489cc2be1a4816b5abace67a136928', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.396629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.396987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.397740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.399524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:09.399524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:09.401673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:09.418414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 814/1659] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:09.503117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.504096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 846/1660] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Jul 28 06:37:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:09.510306 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692fd3-529b-49c7-9652-cbffa6553bbd None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.510598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692fd3-529b-49c7-9652-cbffa6553bbd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.510896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692fd3-529b-49c7-9652-cbffa6553bbd None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.512661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 847/1661] 10.222.0.20 () {54 vars in 912 bytes} [Thu Jul 28 06:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:37:09.517864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6202fc0-b800-4acc-a297-d3621affed9a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.518158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6202fc0-b800-4acc-a297-d3621affed9a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.518438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6202fc0-b800-4acc-a297-d3621affed9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.519555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289867f-af48-4d06-a940-2558bd32a24e tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.526304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c15d942187c4f6fb7cf62df543949c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92489cc2be1a4816b5abace67a136928', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.526965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.527305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.527627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] PATH_INFO: `/v3/projects/bc2779196bed47e0b5cd5eddf62dfba4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.528856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bc2779196bed47e0b5cd5eddf62dfba4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:09.530238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: WARNING py.warnings [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 06:37:09.530238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: warnings.warn(msg) Jul 28 06:37:09.530238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: Jul 28 06:37:09.531170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:09.560586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c6202fc0-b800-4acc-a297-d3621affed9a None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:09.562323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c6202fc0-b800-4acc-a297-d3621affed9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:09.588601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 848/1662] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:37:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:09.602380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6202fc0-b800-4acc-a297-d3621affed9a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:09.612324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc2779196bed47e0b5cd5eddf62dfba4', 'request_id': 'req-8ecf08c4-9b47-4ccd-8758-209c905a07c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.624769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc2779196bed47e0b5cd5eddf62dfba4', 'request_id': 'req-8ecf08c4-9b47-4ccd-8758-209c905a07c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.637215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc2779196bed47e0b5cd5eddf62dfba4', 'request_id': 'req-8ecf08c4-9b47-4ccd-8758-209c905a07c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.670241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] The token cache is being invalidate because project bc2779196bed47e0b5cd5eddf62dfba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:09.671131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.672175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:09.680379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 815/1663] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:09 2022] DELETE /identity/v3/projects/bc2779196bed47e0b5cd5eddf62dfba4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:09.783218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6202fc0-b800-4acc-a297-d3621affed9a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:09.888050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:09.888924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:09.889488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:09.890178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:09.893029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:10.012929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:10.114599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:10.116344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:10.155832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 849/1664] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:10.201816 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:10.203428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:10.351260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:10.352711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:10.436241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf08c4-9b47-4ccd-8758-209c905a07c0 tempest-VolumeQuotaClassesTest-1517422270 tempest-VolumeQuotaClassesTest-1517422270-project-admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:10.452934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:10.455243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.455743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.456206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] PATH_INFO: `/v3/projects/c71fa2391435493e9533ca36af2426f2` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.459208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c71fa2391435493e9533ca36af2426f2)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:10.461453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:10.534610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4107d-9a1e-4be5-b68c-e5e0c1ba4967 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:10.543528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:10.547849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:10.550501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.550789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.552769 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] PATH_INFO: `/v3/users/b51b943f169742f28a9e6222bf04c986` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.557141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51b943f169742f28a9e6222bf04c986)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:10.565397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:10.634202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c71fa2391435493e9533ca36af2426f2', 'request_id': 'req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.655028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c71fa2391435493e9533ca36af2426f2', 'request_id': 'req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.672208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c71fa2391435493e9533ca36af2426f2', 'request_id': 'req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.696986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51b943f169742f28a9e6222bf04c986'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.711679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] The token cache is being invalidate because project c71fa2391435493e9533ca36af2426f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:10.715729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.718216 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.723540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 816/1665] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:10 2022] DELETE /identity/v3/projects/c71fa2391435493e9533ca36af2426f2 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:10.724000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51b943f169742f28a9e6222bf04c986'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.728584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:10.733774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:10.747960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51b943f169742f28a9e6222bf04c986'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.748690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51b943f169742f28a9e6222bf04c986'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.778756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e517a53a-3bea-4855-ba8d-7e2b50f80d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51b943f169742f28a9e6222bf04c986'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:10.865979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 850/1666] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:10 2022] DELETE /identity/v3/users/b51b943f169742f28a9e6222bf04c986 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:10.872365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad0df-a42d-49f6-ae05-76553d2f5276 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:10.872670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad0df-a42d-49f6-ae05-76553d2f5276 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:10.894279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad0df-a42d-49f6-ae05-76553d2f5276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:10.913787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36449c-5bf2-4b8b-ae10-cd85cad58b41 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:10.988467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-366ad0df-a42d-49f6-ae05-76553d2f5276 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:10.998251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-366ad0df-a42d-49f6-ae05-76553d2f5276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:11.158765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:11.159440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:11.159699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:11.160354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] PATH_INFO: `/v3/users/4a902783d9f34ed9988958e90e012ab9` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:11.170249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a902783d9f34ed9988958e90e012ab9)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:11.172232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:11.205216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:11.263773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 851/1667] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:11.329022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a902783d9f34ed9988958e90e012ab9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:11.349918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a902783d9f34ed9988958e90e012ab9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:11.367513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a902783d9f34ed9988958e90e012ab9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:11.372378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ad0df-a42d-49f6-ae05-76553d2f5276 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:11.373875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-366ad0df-a42d-49f6-ae05-76553d2f5276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:11.394092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:11.442200 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a902783d9f34ed9988958e90e012ab9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:11.445835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a902783d9f34ed9988958e90e012ab9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:11.503792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 817/1668] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:10 2022] DELETE /identity/v3/users/4a902783d9f34ed9988958e90e012ab9 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:11.590972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ad0df-a42d-49f6-ae05-76553d2f5276 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:11.791993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc093e1-88aa-44f3-a48c-e1bdb102e157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:11.792857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc093e1-88aa-44f3-a48c-e1bdb102e157 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:11.793205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc093e1-88aa-44f3-a48c-e1bdb102e157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:11.810497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc093e1-88aa-44f3-a48c-e1bdb102e157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:11.813070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fcc093e1-88aa-44f3-a48c-e1bdb102e157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:11.957129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc093e1-88aa-44f3-a48c-e1bdb102e157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:11.959189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc093e1-88aa-44f3-a48c-e1bdb102e157 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:12.002669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 852/1669] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:13.330626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:13.342305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:13.395744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c8b1b-7ff8-4c03-8561-ded5d42b4fa6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:13.422524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:13.424050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:13.426093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:13.426395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] PATH_INFO: `/v3/projects/009965e800bc4eef8def39a5cd382ab4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:13.431842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009965e800bc4eef8def39a5cd382ab4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:13.439636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:13.593531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009965e800bc4eef8def39a5cd382ab4', 'request_id': 'req-eb91aa18-c2aa-4fa7-be88-d901a95463a8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:13.604909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009965e800bc4eef8def39a5cd382ab4', 'request_id': 'req-eb91aa18-c2aa-4fa7-be88-d901a95463a8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:13.613424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009965e800bc4eef8def39a5cd382ab4', 'request_id': 'req-eb91aa18-c2aa-4fa7-be88-d901a95463a8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:13.638316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] The token cache is being invalidate because project 009965e800bc4eef8def39a5cd382ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:13.639203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:13.639751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-eb91aa18-c2aa-4fa7-be88-d901a95463a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:13.642800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 818/1670] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:13 2022] DELETE /identity/v3/projects/009965e800bc4eef8def39a5cd382ab4 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:13.649056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105687-d346-4421-947a-1c2e7074b454 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:13.650226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105687-d346-4421-947a-1c2e7074b454 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:13.650226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105687-d346-4421-947a-1c2e7074b454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:13.698337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-35105687-d346-4421-947a-1c2e7074b454 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:13.699598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-35105687-d346-4421-947a-1c2e7074b454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:13.832192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:13.856778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 853/1671] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:13.864246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc83ac-6a77-4d29-93a6-7ac69fb7f18f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:13.865083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc83ac-6a77-4d29-93a6-7ac69fb7f18f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:13.868861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc83ac-6a77-4d29-93a6-7ac69fb7f18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:13.915723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1ffc83ac-6a77-4d29-93a6-7ac69fb7f18f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:13.917102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1ffc83ac-6a77-4d29-93a6-7ac69fb7f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:13.932386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 819/1672] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:13.934013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35105687-d346-4421-947a-1c2e7074b454 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:13.951875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35105687-d346-4421-947a-1c2e7074b454 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:13.970603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:13.971356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:13.971710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:13.973816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:13.975381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:13.977589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:13.988013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:14.037555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 854/1673] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:14.039962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc83ac-6a77-4d29-93a6-7ac69fb7f18f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:14.057191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc83ac-6a77-4d29-93a6-7ac69fb7f18f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:14.070336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:14.071556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:14.071946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:14.072362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:14.073284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:14.075513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:14.127657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 820/1674] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:14 2022] POST /identity/v3/projects => generated 367 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:14.133330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:14.146744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93b952-6439-44a8-a632-085334253f4a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:14.256551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0382ec-22d8-40a2-be70-7a07b32fcf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:14.257260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0382ec-22d8-40a2-be70-7a07b32fcf16 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:14.257608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0382ec-22d8-40a2-be70-7a07b32fcf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:14.257891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0382ec-22d8-40a2-be70-7a07b32fcf16 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:14.258571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0382ec-22d8-40a2-be70-7a07b32fcf16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:14.260696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0382ec-22d8-40a2-be70-7a07b32fcf16 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:14.293775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:14.296215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:14.303650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 855/1675] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:14 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:14.305025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0382ec-22d8-40a2-be70-7a07b32fcf16 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:14.324178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605c4ed-6839-49f6-9fe6-e2335b3fdade admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:14.328633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0382ec-22d8-40a2-be70-7a07b32fcf16 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:14.336538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f8df7-3450-4b48-8350-01e3040aae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:14.337457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f8df7-3450-4b48-8350-01e3040aae14 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:14.337716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f8df7-3450-4b48-8350-01e3040aae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:14.338421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f8df7-3450-4b48-8350-01e3040aae14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:14.338932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:14.339407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f8df7-3450-4b48-8350-01e3040aae14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:14.340592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:14.341749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f8df7-3450-4b48-8350-01e3040aae14 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:14.343857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:14.344400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] PATH_INFO: `/v3/projects/1b2913c373d244a2987135e3dc4cdac4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:14.347964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b2913c373d244a2987135e3dc4cdac4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:14.350671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:14.363340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 856/1676] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:14.365496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f8df7-3450-4b48-8350-01e3040aae14 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:14.394754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f8df7-3450-4b48-8350-01e3040aae14 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:14.404328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1739a-c222-4a9d-8b37-c7ce8df0b5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:14.405100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1739a-c222-4a9d-8b37-c7ce8df0b5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:14.405625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1739a-c222-4a9d-8b37-c7ce8df0b5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:14.405934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1739a-c222-4a9d-8b37-c7ce8df0b5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:14.406717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1739a-c222-4a9d-8b37-c7ce8df0b5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:14.409949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1739a-c222-4a9d-8b37-c7ce8df0b5d4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:14.430105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 857/1677] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:14.438637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1739a-c222-4a9d-8b37-c7ce8df0b5d4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:14.500206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1739a-c222-4a9d-8b37-c7ce8df0b5d4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:14.503053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b2913c373d244a2987135e3dc4cdac4', 'request_id': 'req-aa263c09-cb30-4896-b24a-22c7d7f6c923'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:14.521160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b2913c373d244a2987135e3dc4cdac4', 'request_id': 'req-aa263c09-cb30-4896-b24a-22c7d7f6c923'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:14.531048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b2913c373d244a2987135e3dc4cdac4', 'request_id': 'req-aa263c09-cb30-4896-b24a-22c7d7f6c923'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:14.550515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] The token cache is being invalidate because project 1b2913c373d244a2987135e3dc4cdac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:14.550809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:14.556092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:14.558228 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 821/1678] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:14 2022] DELETE /identity/v3/projects/1b2913c373d244a2987135e3dc4cdac4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:14.562151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:14.564104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:14.629710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:14.714050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19d333-ab3f-4e31-9bbe-56ff9eb8be81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:14.717156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19d333-ab3f-4e31-9bbe-56ff9eb8be81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:14.730957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19d333-ab3f-4e31-9bbe-56ff9eb8be81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:14.731812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19d333-ab3f-4e31-9bbe-56ff9eb8be81 admin admin] PATH_INFO: `/v3/projects/87658d48295748fda5c550debf8956f5/users/a8f6b5d3b24646718029b862e22fe7f7/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:14.779579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19d333-ab3f-4e31-9bbe-56ff9eb8be81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87658d48295748fda5c550debf8956f5, user_id=a8f6b5d3b24646718029b862e22fe7f7, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:14.784314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19d333-ab3f-4e31-9bbe-56ff9eb8be81 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:14.830133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 858/1679] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:14 2022] PUT /identity/v3/projects/87658d48295748fda5c550debf8956f5/users/a8f6b5d3b24646718029b862e22fe7f7/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:14.832004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1a258-2fc2-4017-8d10-f8df03d444ed None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:14.832426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1a258-2fc2-4017-8d10-f8df03d444ed None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:14.832826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1a258-2fc2-4017-8d10-f8df03d444ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:14.858419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa263c09-cb30-4896-b24a-22c7d7f6c923 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:14.897776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-26d1a258-2fc2-4017-8d10-f8df03d444ed None None] MFA Rules not processed for user `a8f6b5d3b24646718029b862e22fe7f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:14.899401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-26d1a258-2fc2-4017-8d10-f8df03d444ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:14.970813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:15.050246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 859/1680] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:15.051252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1a258-2fc2-4017-8d10-f8df03d444ed None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:15.053740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-26d1a258-2fc2-4017-8d10-f8df03d444ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:15.097280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:15.100901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:15.101206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:15.102694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] PATH_INFO: `/v3/users/9c15d942187c4f6fb7cf62df543949c8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:15.107835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c15d942187c4f6fb7cf62df543949c8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:15.111670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:15.157053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1a258-2fc2-4017-8d10-f8df03d444ed None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:15.164853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c02b5-ea7f-4500-95e7-94c84ce0db78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:15.165531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c02b5-ea7f-4500-95e7-94c84ce0db78 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:15.165798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c02b5-ea7f-4500-95e7-94c84ce0db78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:15.166067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c02b5-ea7f-4500-95e7-94c84ce0db78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:15.166750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c02b5-ea7f-4500-95e7-94c84ce0db78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:15.170721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c02b5-ea7f-4500-95e7-94c84ce0db78 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:15.220916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 860/1681] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:15 2022] POST /identity/v3/projects => generated 369 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:15.222044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c02b5-ea7f-4500-95e7-94c84ce0db78 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:15.283859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c15d942187c4f6fb7cf62df543949c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:15.305473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c15d942187c4f6fb7cf62df543949c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:15.306386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c15d942187c4f6fb7cf62df543949c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:15.335039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c15d942187c4f6fb7cf62df543949c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:15.336627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c15d942187c4f6fb7cf62df543949c8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:15.360887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c02b5-ea7f-4500-95e7-94c84ce0db78 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:15.366435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1dbf5-9ff2-442d-857f-4f36fb0397da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:15.367250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1dbf5-9ff2-442d-857f-4f36fb0397da admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:15.367535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1dbf5-9ff2-442d-857f-4f36fb0397da admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:15.367807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1dbf5-9ff2-442d-857f-4f36fb0397da admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:15.368655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1dbf5-9ff2-442d-857f-4f36fb0397da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:15.370817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1dbf5-9ff2-442d-857f-4f36fb0397da admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:15.373518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 822/1682] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:14 2022] DELETE /identity/v3/users/9c15d942187c4f6fb7cf62df543949c8 => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:15.415060 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:15.416815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:15.423716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 861/1683] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:15 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:15.506726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:15.531490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e6c28-803d-45c7-993d-4ae7893485be admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:15.538659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b275-84ee-4fde-91ec-04c1ca3d5d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:15.539556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b275-84ee-4fde-91ec-04c1ca3d5d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:15.540689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b275-84ee-4fde-91ec-04c1ca3d5d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:15.541179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b275-84ee-4fde-91ec-04c1ca3d5d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:15.542137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1b275-84ee-4fde-91ec-04c1ca3d5d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:15.544237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1b275-84ee-4fde-91ec-04c1ca3d5d05 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:15.564833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1dbf5-9ff2-442d-857f-4f36fb0397da admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:15.577399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 823/1684] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:15.578581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1dbf5-9ff2-442d-857f-4f36fb0397da admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:15.584614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5760c-f374-4f5e-bd24-56a1032e3704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:15.585492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5760c-f374-4f5e-bd24-56a1032e3704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:15.585949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5760c-f374-4f5e-bd24-56a1032e3704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:15.586293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5760c-f374-4f5e-bd24-56a1032e3704 admin admin] PATH_INFO: `/v3/projects/0b8181ceae4f4fb6a92ea8b21f89228e/users/6bf6cc4fb8cc488b81014984a785c316/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:15.608111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5760c-f374-4f5e-bd24-56a1032e3704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8181ceae4f4fb6a92ea8b21f89228e, user_id=6bf6cc4fb8cc488b81014984a785c316, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:15.611356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5760c-f374-4f5e-bd24-56a1032e3704 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:15.638173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b275-84ee-4fde-91ec-04c1ca3d5d05 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:15.649335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 862/1685] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:15 2022] PUT /identity/v3/projects/0b8181ceae4f4fb6a92ea8b21f89228e/users/6bf6cc4fb8cc488b81014984a785c316/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:15.657820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b275-84ee-4fde-91ec-04c1ca3d5d05 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:15.700756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed585-e25b-4616-a1f2-9f7ba92797e3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:15.701134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed585-e25b-4616-a1f2-9f7ba92797e3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:15.701503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed585-e25b-4616-a1f2-9f7ba92797e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:15.740189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a49ed585-e25b-4616-a1f2-9f7ba92797e3 None None] MFA Rules not processed for user `cf13ae714ad04d6696c273603bba9fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:15.741760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a49ed585-e25b-4616-a1f2-9f7ba92797e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:15.754549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:15.819264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d0397-855c-4cf1-a2f0-adc4b0ff564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:15.820447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d0397-855c-4cf1-a2f0-adc4b0ff564d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:15.820910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d0397-855c-4cf1-a2f0-adc4b0ff564d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:15.821393 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d0397-855c-4cf1-a2f0-adc4b0ff564d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:15.822276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d0397-855c-4cf1-a2f0-adc4b0ff564d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:15.824415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d0397-855c-4cf1-a2f0-adc4b0ff564d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:15.894656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 824/1686] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:15.896469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d0397-855c-4cf1-a2f0-adc4b0ff564d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:15.929898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 863/1687] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:15.937860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d0397-855c-4cf1-a2f0-adc4b0ff564d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:15.941682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ed585-e25b-4616-a1f2-9f7ba92797e3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:15.945663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a49ed585-e25b-4616-a1f2-9f7ba92797e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:15.948157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fd3d9-8404-44ae-83e0-e2f74a0cf5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:15.949105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fd3d9-8404-44ae-83e0-e2f74a0cf5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:15.949634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fd3d9-8404-44ae-83e0-e2f74a0cf5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:15.950098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fd3d9-8404-44ae-83e0-e2f74a0cf5ae admin admin] PATH_INFO: `/v3/projects/0b8181ceae4f4fb6a92ea8b21f89228e/users/6bf6cc4fb8cc488b81014984a785c316/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:15.954208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fd3d9-8404-44ae-83e0-e2f74a0cf5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8181ceae4f4fb6a92ea8b21f89228e, user_id=6bf6cc4fb8cc488b81014984a785c316, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:15.958719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fd3d9-8404-44ae-83e0-e2f74a0cf5ae admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:16.008431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 825/1688] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:15 2022] PUT /identity/v3/projects/0b8181ceae4f4fb6a92ea8b21f89228e/users/6bf6cc4fb8cc488b81014984a785c316/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:16.021146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fc902-0aaf-4f86-8f3d-783b7f54243c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:16.021483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fc902-0aaf-4f86-8f3d-783b7f54243c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:16.024250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fc902-0aaf-4f86-8f3d-783b7f54243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:16.154917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ed585-e25b-4616-a1f2-9f7ba92797e3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:16.196466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9b4fc902-0aaf-4f86-8f3d-783b7f54243c None None] MFA Rules not processed for user `6bf6cc4fb8cc488b81014984a785c316`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:16.197802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9b4fc902-0aaf-4f86-8f3d-783b7f54243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:16.318194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:16.321444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:16.321743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:16.322026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:16.410257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 826/1689] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:16 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:16.429020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fc902-0aaf-4f86-8f3d-783b7f54243c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:16.439855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9b4fc902-0aaf-4f86-8f3d-783b7f54243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:16.458413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:16.493332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:16.495501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:16.516741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 864/1690] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:16.558960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:16.566051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fc902-0aaf-4f86-8f3d-783b7f54243c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:16.572810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:16.574261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:16.574711 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:16.575003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:16.590330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:16.591932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:16.608747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 827/1691] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:16 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:16.695971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:16.697625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:16.718887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc9440-eb3c-405e-a9fc-cacbe5c1f389 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:16.731646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:16.732520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:16.732952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:16.733433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:16.735441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:16.833707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:16.835264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:16.857647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 865/1692] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:17.507120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:17.509987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:17.643333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b90a8-b434-4800-80f1-87a427a7587d service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:17.658542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:17.659220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:17.659500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:17.659772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] PATH_INFO: `/v3/users/6ba9ce6cb5404b358a3c9591a7aa1775` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:17.661195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba9ce6cb5404b358a3c9591a7aa1775)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:17.663728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:17.763037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba9ce6cb5404b358a3c9591a7aa1775'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.779678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba9ce6cb5404b358a3c9591a7aa1775'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.780531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba9ce6cb5404b358a3c9591a7aa1775'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.822224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba9ce6cb5404b358a3c9591a7aa1775'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.823074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-59fc35e4-dc5f-430a-a832-4df6a68ff429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba9ce6cb5404b358a3c9591a7aa1775'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:17.850780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:17.858013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 828/1693] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:17 2022] DELETE /identity/v3/users/6ba9ce6cb5404b358a3c9591a7aa1775 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:17.860090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:17.862473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9a43-ebad-4d15-a1f1-72b0066e05f4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:17.863185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9a43-ebad-4d15-a1f1-72b0066e05f4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:17.863185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9a43-ebad-4d15-a1f1-72b0066e05f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:17.904853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba358955-c168-4141-8685-8d5c66def9fe service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:17.919873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d05b9a43-ebad-4d15-a1f1-72b0066e05f4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:17.920612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d05b9a43-ebad-4d15-a1f1-72b0066e05f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:18.060608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 829/1694] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:18.109435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:18.110369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:18.110814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:18.111254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] PATH_INFO: `/v3/projects/92489cc2be1a4816b5abace67a136928` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:18.112558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92489cc2be1a4816b5abace67a136928)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:18.115106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:18.133329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9a43-ebad-4d15-a1f1-72b0066e05f4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:18.135131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d05b9a43-ebad-4d15-a1f1-72b0066e05f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:18.150898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:18.228748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92489cc2be1a4816b5abace67a136928', 'request_id': 'req-df394154-aebf-4a71-8a86-8e557b30acd4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.249251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92489cc2be1a4816b5abace67a136928', 'request_id': 'req-df394154-aebf-4a71-8a86-8e557b30acd4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.286864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92489cc2be1a4816b5abace67a136928', 'request_id': 'req-df394154-aebf-4a71-8a86-8e557b30acd4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.289938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9a43-ebad-4d15-a1f1-72b0066e05f4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:18.328482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] The token cache is being invalidate because project 92489cc2be1a4816b5abace67a136928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:18.330383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.331452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-df394154-aebf-4a71-8a86-8e557b30acd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:18.339800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 866/1695] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:17 2022] DELETE /identity/v3/projects/92489cc2be1a4816b5abace67a136928 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:18.346321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa68147-78ce-457a-8576-c4e59f90d5da None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:18.348087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa68147-78ce-457a-8576-c4e59f90d5da None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:18.348578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa68147-78ce-457a-8576-c4e59f90d5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:18.424211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0aa68147-78ce-457a-8576-c4e59f90d5da None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:18.427839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0aa68147-78ce-457a-8576-c4e59f90d5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:18.444264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:18.447014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:18.448644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:18.448644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:18.448644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:18.461422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:18.574563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 867/1696] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:18.584366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d207b7-4729-4c44-bb9b-04bba60026e8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:18.584792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d207b7-4729-4c44-bb9b-04bba60026e8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:18.585326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d207b7-4729-4c44-bb9b-04bba60026e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:18.629134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a2d207b7-4729-4c44-bb9b-04bba60026e8 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:18.632675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a2d207b7-4729-4c44-bb9b-04bba60026e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:18.674327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 868/1697] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:18.678655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d207b7-4729-4c44-bb9b-04bba60026e8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:18.699512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:18.701333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:18.702899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d207b7-4729-4c44-bb9b-04bba60026e8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:18.723055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 830/1698] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:18.734196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:18.735087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:18.738097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:18.738634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:18.739644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:18.740164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:18.750892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:18.784398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 869/1699] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:18.785945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:18.798818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082100bc-75f3-43ee-83db-e615fd592f5f service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:18.813112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50c96e-8d05-48ed-950e-b96690798930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:18.813796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50c96e-8d05-48ed-950e-b96690798930 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:18.814081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50c96e-8d05-48ed-950e-b96690798930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:18.814357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50c96e-8d05-48ed-950e-b96690798930 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:18.815509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50c96e-8d05-48ed-950e-b96690798930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:18.828750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50c96e-8d05-48ed-950e-b96690798930 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:18.921352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 831/1700] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:18 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:18.923877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:18.944750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989300b2-0b4a-4637-ab32-502df4ba58fa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:19.103136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba0fc4-6127-4209-a486-7183238c81d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:19.104351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba0fc4-6127-4209-a486-7183238c81d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.104986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba0fc4-6127-4209-a486-7183238c81d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.105505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba0fc4-6127-4209-a486-7183238c81d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.107177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba0fc4-6127-4209-a486-7183238c81d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:19.109542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba0fc4-6127-4209-a486-7183238c81d7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:19.157427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50c96e-8d05-48ed-950e-b96690798930 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:19.158595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 870/1701] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:18 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:19.175119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50c96e-8d05-48ed-950e-b96690798930 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:19.185883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a438df-cfc3-44a2-be00-cdd7ab06f997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:19.186899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a438df-cfc3-44a2-be00-cdd7ab06f997 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.189268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a438df-cfc3-44a2-be00-cdd7ab06f997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.189727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a438df-cfc3-44a2-be00-cdd7ab06f997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.190531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a438df-cfc3-44a2-be00-cdd7ab06f997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:19.193174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a438df-cfc3-44a2-be00-cdd7ab06f997 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:19.221484 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba0fc4-6127-4209-a486-7183238c81d7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:19.233602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 832/1702] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:19.238660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba0fc4-6127-4209-a486-7183238c81d7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:19.258894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735dbb7-d913-44da-90de-2f8609d2a5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:19.259938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735dbb7-d913-44da-90de-2f8609d2a5ef admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.260380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735dbb7-d913-44da-90de-2f8609d2a5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.260893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735dbb7-d913-44da-90de-2f8609d2a5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.263188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735dbb7-d913-44da-90de-2f8609d2a5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:19.266091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735dbb7-d913-44da-90de-2f8609d2a5ef admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:19.293539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 871/1703] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:19.295236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a438df-cfc3-44a2-be00-cdd7ab06f997 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:19.311982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a438df-cfc3-44a2-be00-cdd7ab06f997 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:19.337647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:19.339132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.339132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.339427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] PATH_INFO: `/v3/projects/db906862ca974657a0539df041c7e952/users/de8001f410884446a391c7d60f7cd02b/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.382260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] RBAC: Authorizing `identity:create_grant(project_id=db906862ca974657a0539df041c7e952, user_id=de8001f410884446a391c7d60f7cd02b, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:19.397815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:19.441527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 833/1704] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:19 2022] PUT /identity/v3/projects/db906862ca974657a0539df041c7e952/users/de8001f410884446a391c7d60f7cd02b/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:19.454670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7aeef-5de7-4681-83ee-5010a43cf262 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.454962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7aeef-5de7-4681-83ee-5010a43cf262 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.455247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7aeef-5de7-4681-83ee-5010a43cf262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.529226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:19.531196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:19.541242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-06b7aeef-5de7-4681-83ee-5010a43cf262 None None] MFA Rules not processed for user `de8001f410884446a391c7d60f7cd02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:19.542535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-06b7aeef-5de7-4681-83ee-5010a43cf262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:19.586732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:19.750229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a609b2c-c275-407a-a243-7c77553afafe admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:19.767286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:19.768775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:19.770309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.771735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.772043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] PATH_INFO: `/v3/projects/a13254920b734a24948b0bdcad37f37b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.774687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a13254920b734a24948b0bdcad37f37b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:19.781330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 872/1705] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:19.783459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:19.783573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7aeef-5de7-4681-83ee-5010a43cf262 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:19.807947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7aeef-5de7-4681-83ee-5010a43cf262 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:19.836644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f50a-6e50-4e95-b3c2-1ed5af3473ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:19.837406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f50a-6e50-4e95-b3c2-1ed5af3473ba admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:19.837731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f50a-6e50-4e95-b3c2-1ed5af3473ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:19.837969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f50a-6e50-4e95-b3c2-1ed5af3473ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:19.839106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102f50a-6e50-4e95-b3c2-1ed5af3473ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:19.846195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102f50a-6e50-4e95-b3c2-1ed5af3473ba admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:19.932948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 873/1706] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:19 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:19.934454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a13254920b734a24948b0bdcad37f37b', 'request_id': 'req-30988319-fdd9-454a-9efb-68c088458f29'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:19.937832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f50a-6e50-4e95-b3c2-1ed5af3473ba admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:19.974748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a13254920b734a24948b0bdcad37f37b', 'request_id': 'req-30988319-fdd9-454a-9efb-68c088458f29'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.002306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a13254920b734a24948b0bdcad37f37b', 'request_id': 'req-30988319-fdd9-454a-9efb-68c088458f29'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.037489 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] The token cache is being invalidate because project a13254920b734a24948b0bdcad37f37b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:20.037792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.039659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-30988319-fdd9-454a-9efb-68c088458f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:20.046375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 834/1707] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:19 2022] DELETE /identity/v3/projects/a13254920b734a24948b0bdcad37f37b => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:20.053869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758c99-7b75-441c-933e-20bc5b3fa304 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.054206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758c99-7b75-441c-933e-20bc5b3fa304 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.056838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758c99-7b75-441c-933e-20bc5b3fa304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.119950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e4758c99-7b75-441c-933e-20bc5b3fa304 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:20.123033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e4758c99-7b75-441c-933e-20bc5b3fa304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:20.163870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f50a-6e50-4e95-b3c2-1ed5af3473ba admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.183615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.184267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.184551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.184828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.185540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.191958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.246672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 874/1708] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:19 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:20.249457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.251876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:20.285831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bcc4e-e545-4581-8e10-46158242ab01 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.311748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 835/1709] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:20.311953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ff02a-d5cd-4884-a244-24a69de719dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.312659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ff02a-d5cd-4884-a244-24a69de719dc admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.313044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ff02a-d5cd-4884-a244-24a69de719dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.313230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ff02a-d5cd-4884-a244-24a69de719dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.313990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ff02a-d5cd-4884-a244-24a69de719dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.314422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1bf8f-72e0-4b00-a037-3eff891069fc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.314858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1bf8f-72e0-4b00-a037-3eff891069fc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.315240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1bf8f-72e0-4b00-a037-3eff891069fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.315619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ff02a-d5cd-4884-a244-24a69de719dc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.350028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 875/1710] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:20.353549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ff02a-d5cd-4884-a244-24a69de719dc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.371655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ff02a-d5cd-4884-a244-24a69de719dc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.384002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-34a1bf8f-72e0-4b00-a037-3eff891069fc None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:20.385273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-34a1bf8f-72e0-4b00-a037-3eff891069fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:20.388002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851a3a5-53ca-4f03-949f-9d4efc97547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.388676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a3a5-53ca-4f03-949f-9d4efc97547a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.389065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a3a5-53ca-4f03-949f-9d4efc97547a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.389383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a3a5-53ca-4f03-949f-9d4efc97547a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.390101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851a3a5-53ca-4f03-949f-9d4efc97547a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.391696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851a3a5-53ca-4f03-949f-9d4efc97547a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.415574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 836/1711] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:20.417067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1bf8f-72e0-4b00-a037-3eff891069fc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.425586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 876/1712] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:20.429828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851a3a5-53ca-4f03-949f-9d4efc97547a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.435872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1bf8f-72e0-4b00-a037-3eff891069fc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.441848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.443402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.443710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.443974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.445128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:20.445531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.447238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.467384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 837/1713] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:20.473094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.484062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851a3a5-53ca-4f03-949f-9d4efc97547a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.500561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37fcd6-f815-4302-82dd-663a1aa8cf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.501262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37fcd6-f815-4302-82dd-663a1aa8cf96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.502203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37fcd6-f815-4302-82dd-663a1aa8cf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.502594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37fcd6-f815-4302-82dd-663a1aa8cf96 admin admin] PATH_INFO: `/v3/projects/c5dec23576874ca880d454ee34c8feef/users/1cf9948fc8094fe4832882eeb05eb46c/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.511433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227b012-a290-403a-b97e-cab6a19fc9c8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.525906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254ffb6-f93f-4a3b-aa7b-abcc3652621c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.526550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254ffb6-f93f-4a3b-aa7b-abcc3652621c admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.526835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254ffb6-f93f-4a3b-aa7b-abcc3652621c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.527110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254ffb6-f93f-4a3b-aa7b-abcc3652621c admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.527789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254ffb6-f93f-4a3b-aa7b-abcc3652621c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.529088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37fcd6-f815-4302-82dd-663a1aa8cf96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5dec23576874ca880d454ee34c8feef, user_id=1cf9948fc8094fe4832882eeb05eb46c, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.529746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254ffb6-f93f-4a3b-aa7b-abcc3652621c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.534108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37fcd6-f815-4302-82dd-663a1aa8cf96 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.574602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 877/1714] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:20 2022] PUT /identity/v3/projects/c5dec23576874ca880d454ee34c8feef/users/1cf9948fc8094fe4832882eeb05eb46c/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:20.589197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a621fc4-7a1e-41e1-9e48-ba85d30913c0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.589548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a621fc4-7a1e-41e1-9e48-ba85d30913c0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.593247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a621fc4-7a1e-41e1-9e48-ba85d30913c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.595468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 838/1715] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:20 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:20.600287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254ffb6-f93f-4a3b-aa7b-abcc3652621c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.613673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:20.685611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1a621fc4-7a1e-41e1-9e48-ba85d30913c0 None None] MFA Rules not processed for user `1cf9948fc8094fe4832882eeb05eb46c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:20.687473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1a621fc4-7a1e-41e1-9e48-ba85d30913c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:20.698979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254ffb6-f93f-4a3b-aa7b-abcc3652621c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.706363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666c62f-c8e2-4408-8678-5fea5038a917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.706779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c62f-c8e2-4408-8678-5fea5038a917 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.707387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c62f-c8e2-4408-8678-5fea5038a917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.707678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c62f-c8e2-4408-8678-5fea5038a917 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.709551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666c62f-c8e2-4408-8678-5fea5038a917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.711498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666c62f-c8e2-4408-8678-5fea5038a917 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.749143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 839/1716] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:20 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:20.753613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666c62f-c8e2-4408-8678-5fea5038a917 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.779565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666c62f-c8e2-4408-8678-5fea5038a917 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.789046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bc771-326d-415b-9ac8-6d3ea7b0f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.789782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bc771-326d-415b-9ac8-6d3ea7b0f2db admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.790539 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bc771-326d-415b-9ac8-6d3ea7b0f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.790832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bc771-326d-415b-9ac8-6d3ea7b0f2db admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.792311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7bc771-326d-415b-9ac8-6d3ea7b0f2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.796814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7bc771-326d-415b-9ac8-6d3ea7b0f2db admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.836720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 840/1717] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:20.839683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bc771-326d-415b-9ac8-6d3ea7b0f2db admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.851163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 878/1718] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:20.853332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce1bfa-ff97-4c60-8102-c9a52a239e7e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.853869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce1bfa-ff97-4c60-8102-c9a52a239e7e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.854578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce1bfa-ff97-4c60-8102-c9a52a239e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.871935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bc771-326d-415b-9ac8-6d3ea7b0f2db admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.888369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c1bff-20a5-4f85-bdcd-d885fb752ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.888369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1bff-20a5-4f85-bdcd-d885fb752ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.888792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1bff-20a5-4f85-bdcd-d885fb752ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.889204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1bff-20a5-4f85-bdcd-d885fb752ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.893062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c1bff-20a5-4f85-bdcd-d885fb752ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.894715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c1bff-20a5-4f85-bdcd-d885fb752ae4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:20.914089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:20.932388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 841/1719] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:20.933516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c1bff-20a5-4f85-bdcd-d885fb752ae4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:20.941202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-64ce1bfa-ff97-4c60-8102-c9a52a239e7e None None] MFA Rules not processed for user `a2080efc6cbd471995593012bc268765`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:20.945010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-64ce1bfa-ff97-4c60-8102-c9a52a239e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:20.964541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c1bff-20a5-4f85-bdcd-d885fb752ae4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:20.985052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee05968-7450-4906-8e77-174efe653b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:20.985249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee05968-7450-4906-8e77-174efe653b96 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:20.985523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee05968-7450-4906-8e77-174efe653b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:20.986966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee05968-7450-4906-8e77-174efe653b96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:20.989744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee05968-7450-4906-8e77-174efe653b96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:20.994412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee05968-7450-4906-8e77-174efe653b96 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:21.067994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 842/1720] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:20 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:21.072674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee05968-7450-4906-8e77-174efe653b96 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:21.147847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 879/1721] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:21.150471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce1bfa-ff97-4c60-8102-c9a52a239e7e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:21.189990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce1bfa-ff97-4c60-8102-c9a52a239e7e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:21.299845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee05968-7450-4906-8e77-174efe653b96 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:21.316217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa5f2e-738c-4680-a030-224132a3009e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:21.316911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa5f2e-738c-4680-a030-224132a3009e admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:21.317361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa5f2e-738c-4680-a030-224132a3009e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:21.318481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa5f2e-738c-4680-a030-224132a3009e admin admin] PATH_INFO: `/v3/projects/773c6f93b58a42a48700a32a2ce4eb47/users/b7b5998f6af34e289afb6543a1bc3cc6/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:21.373206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa5f2e-738c-4680-a030-224132a3009e admin admin] RBAC: Authorizing `identity:create_grant(project_id=773c6f93b58a42a48700a32a2ce4eb47, user_id=b7b5998f6af34e289afb6543a1bc3cc6, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:21.377877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae83faa-cb5b-4970-b470-0d9babd0b1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:21.378771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae83faa-cb5b-4970-b470-0d9babd0b1bc admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:21.379224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae83faa-cb5b-4970-b470-0d9babd0b1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:21.379692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae83faa-cb5b-4970-b470-0d9babd0b1bc admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:21.380364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa5f2e-738c-4680-a030-224132a3009e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:21.381216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae83faa-cb5b-4970-b470-0d9babd0b1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:21.385495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae83faa-cb5b-4970-b470-0d9babd0b1bc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:21.423653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 843/1722] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:21 2022] PUT /identity/v3/projects/773c6f93b58a42a48700a32a2ce4eb47/users/b7b5998f6af34e289afb6543a1bc3cc6/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:21.425458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0d76f-267c-4336-aed9-e6bdf9c60b84 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:21.425784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0d76f-267c-4336-aed9-e6bdf9c60b84 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:21.427511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0d76f-267c-4336-aed9-e6bdf9c60b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:21.467465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 880/1723] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:21 2022] POST /identity/v3/users => generated 300 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:21.469021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b46fe-b6f2-4cff-b931-44026d6aec30 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:21.473091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b46fe-b6f2-4cff-b931-44026d6aec30 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:21.473725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b46fe-b6f2-4cff-b931-44026d6aec30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:21.510134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-97d0d76f-267c-4336-aed9-e6bdf9c60b84 None None] MFA Rules not processed for user `b240f68acc734bd492843ad4ac35807c`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:21.512975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-97d0d76f-267c-4336-aed9-e6bdf9c60b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:21.535902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-db8b46fe-b6f2-4cff-b931-44026d6aec30 None None] MFA Rules not processed for user `a2080efc6cbd471995593012bc268765`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:21.538997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-db8b46fe-b6f2-4cff-b931-44026d6aec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:21.597805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:21.681861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 881/1724] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:21.683595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa719-201f-4c99-adfd-9a53c9d4ba29 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:21.683896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa719-201f-4c99-adfd-9a53c9d4ba29 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:21.684224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa719-201f-4c99-adfd-9a53c9d4ba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:21.744804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8d7aa719-201f-4c99-adfd-9a53c9d4ba29 None None] MFA Rules not processed for user `b7b5998f6af34e289afb6543a1bc3cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:21.748350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8d7aa719-201f-4c99-adfd-9a53c9d4ba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:21.789337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 844/1725] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:21.789337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0d76f-267c-4336-aed9-e6bdf9c60b84 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:21.815050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:21.965383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 882/1726] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:21.968180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aa719-201f-4c99-adfd-9a53c9d4ba29 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:21.973351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8d7aa719-201f-4c99-adfd-9a53c9d4ba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:21.991565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0d76f-267c-4336-aed9-e6bdf9c60b84 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:22.011968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe950cf-ab71-4c19-bf7c-ef0f5448a1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:22.012318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe950cf-ab71-4c19-bf7c-ef0f5448a1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.015575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe950cf-ab71-4c19-bf7c-ef0f5448a1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.015884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe950cf-ab71-4c19-bf7c-ef0f5448a1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.016764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe950cf-ab71-4c19-bf7c-ef0f5448a1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:22.029879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe950cf-ab71-4c19-bf7c-ef0f5448a1ea admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:22.060740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 845/1727] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:22.064592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ca0c-83bd-4857-af43-f52c2aa659a4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.064965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ca0c-83bd-4857-af43-f52c2aa659a4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.065252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ca0c-83bd-4857-af43-f52c2aa659a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.092127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cc79ca0c-83bd-4857-af43-f52c2aa659a4 None None] MFA Rules not processed for user `cf13ae714ad04d6696c273603bba9fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:22.093413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cc79ca0c-83bd-4857-af43-f52c2aa659a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:22.169361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aa719-201f-4c99-adfd-9a53c9d4ba29 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:22.206254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:22.207177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.207718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.209597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.221499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 846/1728] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:22.225797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:22.236435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c8165-8fcb-43e3-a913-a6f378da292f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.236772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c8165-8fcb-43e3-a913-a6f378da292f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.237060 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c8165-8fcb-43e3-a913-a6f378da292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.238688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:22.272774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f68c8165-8fcb-43e3-a913-a6f378da292f None None] MFA Rules not processed for user `b240f68acc734bd492843ad4ac35807c`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:22.274356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f68c8165-8fcb-43e3-a913-a6f378da292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:22.278463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 883/1729] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:22.279126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:22.280772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:22.285149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 847/1730] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:22.287005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c8165-8fcb-43e3-a913-a6f378da292f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:22.314322 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c8165-8fcb-43e3-a913-a6f378da292f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:22.332028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088dd8f-923b-47c9-9dd8-ce3121df0b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:22.335092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088dd8f-923b-47c9-9dd8-ce3121df0b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.335396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088dd8f-923b-47c9-9dd8-ce3121df0b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.335677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088dd8f-923b-47c9-9dd8-ce3121df0b25 admin admin] PATH_INFO: `/v3/projects/0ac0a495121947bdb152c826e7831839/users/2e0a6262f7774ec2996fa1fa83a10f57/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.385503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088dd8f-923b-47c9-9dd8-ce3121df0b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac0a495121947bdb152c826e7831839, user_id=2e0a6262f7774ec2996fa1fa83a10f57, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:22.399009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088dd8f-923b-47c9-9dd8-ce3121df0b25 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:22.463204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 848/1731] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:22 2022] PUT /identity/v3/projects/0ac0a495121947bdb152c826e7831839/users/2e0a6262f7774ec2996fa1fa83a10f57/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:22.467238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088dd8f-923b-47c9-9dd8-ce3121df0b25 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:22.516297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a991c-b3b7-4725-9126-679d080ab9c1 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:22.621565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:22.675571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088dd8f-923b-47c9-9dd8-ce3121df0b25 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:22.685957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182b161-3b2e-42ad-8998-ec1dd22c51f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:22.687313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182b161-3b2e-42ad-8998-ec1dd22c51f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.687803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182b161-3b2e-42ad-8998-ec1dd22c51f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.688278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182b161-3b2e-42ad-8998-ec1dd22c51f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.689273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182b161-3b2e-42ad-8998-ec1dd22c51f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:22.691337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182b161-3b2e-42ad-8998-ec1dd22c51f3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:22.718866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 849/1732] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:22.723151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63b6c-5173-4741-b63d-331a4a2bd902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:22.725898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63b6c-5173-4741-b63d-331a4a2bd902 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.728504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63b6c-5173-4741-b63d-331a4a2bd902 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.729319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182b161-3b2e-42ad-8998-ec1dd22c51f3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:22.730617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63b6c-5173-4741-b63d-331a4a2bd902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.767352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182b161-3b2e-42ad-8998-ec1dd22c51f3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:22.793108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa11036-d7da-41c6-ad43-e759bcee9ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:22.793938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa11036-d7da-41c6-ad43-e759bcee9ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.794340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa11036-d7da-41c6-ad43-e759bcee9ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.794726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa11036-d7da-41c6-ad43-e759bcee9ca2 admin admin] PATH_INFO: `/v3/projects/0ac0a495121947bdb152c826e7831839/users/2e0a6262f7774ec2996fa1fa83a10f57/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.797885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa11036-d7da-41c6-ad43-e759bcee9ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac0a495121947bdb152c826e7831839, user_id=2e0a6262f7774ec2996fa1fa83a10f57, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:22.802893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:22.803099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa11036-d7da-41c6-ad43-e759bcee9ca2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:22.839831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 850/1733] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:22 2022] PUT /identity/v3/projects/0ac0a495121947bdb152c826e7831839/users/2e0a6262f7774ec2996fa1fa83a10f57/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:22.853728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7ab5-4e10-44b6-a2e7-37f0c004a1f0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:22.854461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7ab5-4e10-44b6-a2e7-37f0c004a1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:22.855674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7ab5-4e10-44b6-a2e7-37f0c004a1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:22.908903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f63b6c-5173-4741-b63d-331a4a2bd902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:22.910417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f63b6c-5173-4741-b63d-331a4a2bd902 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:22.930720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 884/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:22.949079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6ffd7ab5-4e10-44b6-a2e7-37f0c004a1f0 None None] MFA Rules not processed for user `2e0a6262f7774ec2996fa1fa83a10f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:22.952534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6ffd7ab5-4e10-44b6-a2e7-37f0c004a1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:23.158443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 851/1735] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:23.374655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c97c3-3051-412a-b232-ca3d23134d64 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:23.375198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c97c3-3051-412a-b232-ca3d23134d64 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:23.375198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c97c3-3051-412a-b232-ca3d23134d64 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:23.376553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 885/1736] 10.222.0.20 () {54 vars in 912 bytes} [Thu Jul 28 06:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:37:23.384749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129dbe7-7557-4e95-a861-610e7873f44b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:23.385159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129dbe7-7557-4e95-a861-610e7873f44b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:23.385671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129dbe7-7557-4e95-a861-610e7873f44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:23.425614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c97c3-3051-412a-b232-ca3d23134d64 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:23.547723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7129dbe7-7557-4e95-a861-610e7873f44b None None] MFA Rules not processed for user `4b46a15d481f4393bef8c4fa2281710b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:23.553569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7129dbe7-7557-4e95-a861-610e7873f44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:23.620500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c97c3-3051-412a-b232-ca3d23134d64 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:23.633017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:23.634088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:23.634381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:23.634658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:23.659226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:23.664543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:23.700875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 886/1737] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:23.727881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:23.729599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:23.736718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 852/1738] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:37:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:23.750781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aabcd0-1873-48c2-81fd-856432115673 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:23.756658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f164a4c-e344-4ba8-bdf9-4e73fa94b4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:23.757609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f164a4c-e344-4ba8-bdf9-4e73fa94b4bd service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:23.758290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f164a4c-e344-4ba8-bdf9-4e73fa94b4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:23.758732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f164a4c-e344-4ba8-bdf9-4e73fa94b4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:23.771132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f164a4c-e344-4ba8-bdf9-4e73fa94b4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:23.773529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f164a4c-e344-4ba8-bdf9-4e73fa94b4bd service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:23.808257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 887/1739] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:24.171522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57394609-cec9-468e-a315-de8b47176295 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:24.171996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57394609-cec9-468e-a315-de8b47176295 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:24.175641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57394609-cec9-468e-a315-de8b47176295 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:24.178030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 853/1740] 10.222.0.20 () {54 vars in 912 bytes} [Thu Jul 28 06:37:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:37:24.189854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c134dc-5e2b-42d1-b7fd-c90f03e7cc61 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:24.190166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c134dc-5e2b-42d1-b7fd-c90f03e7cc61 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:24.190549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c134dc-5e2b-42d1-b7fd-c90f03e7cc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:24.227152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57394609-cec9-468e-a315-de8b47176295 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:24.229534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-57394609-cec9-468e-a315-de8b47176295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:24.256541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c8c134dc-5e2b-42d1-b7fd-c90f03e7cc61 None None] MFA Rules not processed for user `4b46a15d481f4393bef8c4fa2281710b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:24.267007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c8c134dc-5e2b-42d1-b7fd-c90f03e7cc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:24.309804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 888/1741] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:37:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:24.348109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:24.406758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57394609-cec9-468e-a315-de8b47176295 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:24.431396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:24.432763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:24.435664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:24.436043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] PATH_INFO: `/v3/users/a8f6b5d3b24646718029b862e22fe7f7` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:24.438055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f6b5d3b24646718029b862e22fe7f7)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:24.440168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:24.449925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c134dc-5e2b-42d1-b7fd-c90f03e7cc61 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:24.453071 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c8c134dc-5e2b-42d1-b7fd-c90f03e7cc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:24.485457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c134dc-5e2b-42d1-b7fd-c90f03e7cc61 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:24.501709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:24.502277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:24.504341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:24.504774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:24.556381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:24.559113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:24.586043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f6b5d3b24646718029b862e22fe7f7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:24.607900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f6b5d3b24646718029b862e22fe7f7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:24.608272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 889/1742] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:24.609037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f6b5d3b24646718029b862e22fe7f7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:24.612527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:24.615458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:24.638810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f6b5d3b24646718029b862e22fe7f7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:24.640067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50c9128-6f5c-4183-a92a-1c38d9a74558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f6b5d3b24646718029b862e22fe7f7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:24.659144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 854/1743] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:24 2022] DELETE /identity/v3/users/a8f6b5d3b24646718029b862e22fe7f7 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:24.664611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade6df-a261-4409-b6d5-bd36dfb8fc4b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:24.666040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade6df-a261-4409-b6d5-bd36dfb8fc4b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:24.666814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade6df-a261-4409-b6d5-bd36dfb8fc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:24.717511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8cade6df-a261-4409-b6d5-bd36dfb8fc4b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:24.719089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8cade6df-a261-4409-b6d5-bd36dfb8fc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:24.838651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b230860-7738-445f-8f92-1f92729193e6 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:24.859391 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 855/1744] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:24.860956 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26351ca3-4a13-4c6f-a1a1-ee9be73388fb None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:24.865238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26351ca3-4a13-4c6f-a1a1-ee9be73388fb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:24.867213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26351ca3-4a13-4c6f-a1a1-ee9be73388fb None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:24.875330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 856/1745] 10.222.0.20 () {54 vars in 961 bytes} [Thu Jul 28 06:37:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:37:24.876285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26351ca3-4a13-4c6f-a1a1-ee9be73388fb None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:24.880898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-26351ca3-4a13-4c6f-a1a1-ee9be73388fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:24.954997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:25.036762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:25.038231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:25.040076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:25.040734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:25.045157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:25.045753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26351ca3-4a13-4c6f-a1a1-ee9be73388fb None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:25.054851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b881ee-b1c6-42eb-9eca-36382daad00a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:25.055579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b881ee-b1c6-42eb-9eca-36382daad00a service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:25.056636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b881ee-b1c6-42eb-9eca-36382daad00a service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:25.056929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b881ee-b1c6-42eb-9eca-36382daad00a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:25.075161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b881ee-b1c6-42eb-9eca-36382daad00a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:25.076009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b881ee-b1c6-42eb-9eca-36382daad00a service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:25.103465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 857/1746] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:25.104885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3700-ee0b-4262-8e23-c348683ddbab None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:25.107592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3700-ee0b-4262-8e23-c348683ddbab None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:25.107592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3700-ee0b-4262-8e23-c348683ddbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:25.123529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:25.182648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-15fc3700-ee0b-4262-8e23-c348683ddbab None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:25.183919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-15fc3700-ee0b-4262-8e23-c348683ddbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:25.231290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:25.238090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:25.262264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 890/1747] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:25.412641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 858/1748] 10.222.0.20 () {58 vars in 1058 bytes} [Thu Jul 28 06:37:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:25.413669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:25.415520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:25.435990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc3700-ee0b-4262-8e23-c348683ddbab None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:25.463460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc3700-ee0b-4262-8e23-c348683ddbab None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:25.477115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3982c-c404-4811-ad8f-6c58d76b18dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:25.478788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3982c-c404-4811-ad8f-6c58d76b18dd service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:25.480298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3982c-c404-4811-ad8f-6c58d76b18dd service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:25.480298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3982c-c404-4811-ad8f-6c58d76b18dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:25.481759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d6c3982c-c404-4811-ad8f-6c58d76b18dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:25.503355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3982c-c404-4811-ad8f-6c58d76b18dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:25.506532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3982c-c404-4811-ad8f-6c58d76b18dd service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:25.522937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 859/1749] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:25.552854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e991838-5560-4aa8-9a5e-a67da48704d5 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:25.558814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:25.560002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:25.560503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:25.561642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:25.564603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:25.800525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:25.803802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:25.842147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 891/1750] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:26.490776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978f93-86cd-4867-a865-ffaa3198e7a6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:26.491314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978f93-86cd-4867-a865-ffaa3198e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:26.491314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978f93-86cd-4867-a865-ffaa3198e7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:26.611396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:26.613591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:26.653737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5aa2fa-b2e5-4d76-b6bf-816758cb91c2 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:26.663196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cc978f93-86cd-4867-a865-ffaa3198e7a6 None None] MFA Rules not processed for user `4a1db92e28444854b9de817dd36104ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:26.666503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cc978f93-86cd-4867-a865-ffaa3198e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:26.673174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:26.674549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:26.677966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:26.680407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] PATH_INFO: `/v3/projects/87658d48295748fda5c550debf8956f5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:26.684046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87658d48295748fda5c550debf8956f5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:26.690964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:26.765788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:26.918029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87658d48295748fda5c550debf8956f5', 'request_id': 'req-e78d1079-9203-4fbe-a658-a6637ea41525'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:26.936866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87658d48295748fda5c550debf8956f5', 'request_id': 'req-e78d1079-9203-4fbe-a658-a6637ea41525'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:26.971909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87658d48295748fda5c550debf8956f5', 'request_id': 'req-e78d1079-9203-4fbe-a658-a6637ea41525'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:27.010242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] The token cache is being invalidate because project 87658d48295748fda5c550debf8956f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:27.010553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:27.013376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:27.016502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 892/1751] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:26 2022] DELETE /identity/v3/projects/87658d48295748fda5c550debf8956f5 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:27.020875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:27.025894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:27.042573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 860/1752] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 556 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:27.240604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d1079-9203-4fbe-a658-a6637ea41525 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:27.267592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:27.268415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:27.268799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:27.269525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] PATH_INFO: `/v3/users/6bf6cc4fb8cc488b81014984a785c316` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:27.271078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bf6cc4fb8cc488b81014984a785c316)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:27.273147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:27.475361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bf6cc4fb8cc488b81014984a785c316'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:27.508416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bf6cc4fb8cc488b81014984a785c316'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:27.512449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978f93-86cd-4867-a865-ffaa3198e7a6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:27.519588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cc978f93-86cd-4867-a865-ffaa3198e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:27.527578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bf6cc4fb8cc488b81014984a785c316'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:27.529812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bf6cc4fb8cc488b81014984a785c316'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:27.555098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d52e8dbd-ad12-4f46-ae0f-0e66e9ba944d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bf6cc4fb8cc488b81014984a785c316'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:27.578661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 893/1753] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:27 2022] DELETE /identity/v3/users/6bf6cc4fb8cc488b81014984a785c316 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:27.581864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652bed-4efe-492b-84fe-3563d4cae0ba None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:27.582277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652bed-4efe-492b-84fe-3563d4cae0ba None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:27.582388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978f93-86cd-4867-a865-ffaa3198e7a6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:27.582504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652bed-4efe-492b-84fe-3563d4cae0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:27.630642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-14652bed-4efe-492b-84fe-3563d4cae0ba None None] MFA Rules not processed for user `a2080efc6cbd471995593012bc268765`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:27.633803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-14652bed-4efe-492b-84fe-3563d4cae0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:27.658170 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:27.757268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 894/1754] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:27.873072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2698c7-1ad5-41bb-979c-039f508288a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:27.875342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2698c7-1ad5-41bb-979c-039f508288a1 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:27.875342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2698c7-1ad5-41bb-979c-039f508288a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:27.875631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2698c7-1ad5-41bb-979c-039f508288a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:27.886298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4b2698c7-1ad5-41bb-979c-039f508288a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:27.929504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:28.146859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2698c7-1ad5-41bb-979c-039f508288a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:28.148514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2698c7-1ad5-41bb-979c-039f508288a1 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:28.196660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 861/1755] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:28.467202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14652bed-4efe-492b-84fe-3563d4cae0ba None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:28.470094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-14652bed-4efe-492b-84fe-3563d4cae0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:28.721269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14652bed-4efe-492b-84fe-3563d4cae0ba None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:28.730422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:28.730999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:28.732304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:28.733256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] PATH_INFO: `/v3/projects/0b8181ceae4f4fb6a92ea8b21f89228e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:28.735872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8181ceae4f4fb6a92ea8b21f89228e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:28.743969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:28.900300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8181ceae4f4fb6a92ea8b21f89228e', 'request_id': 'req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:28.935521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8181ceae4f4fb6a92ea8b21f89228e', 'request_id': 'req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:28.959253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8181ceae4f4fb6a92ea8b21f89228e', 'request_id': 'req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:29.004591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] The token cache is being invalidate because project 0b8181ceae4f4fb6a92ea8b21f89228e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:29.005381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:29.006610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6450b19a-8fcc-4b97-8d1a-8d58e357bb37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:29.010522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 895/1756] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:28 2022] DELETE /identity/v3/projects/0b8181ceae4f4fb6a92ea8b21f89228e => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:29.015545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78dbeb-ae3d-4d82-8736-29f729425466 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:29.016111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78dbeb-ae3d-4d82-8736-29f729425466 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:29.016281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78dbeb-ae3d-4d82-8736-29f729425466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:29.091653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1e78dbeb-ae3d-4d82-8736-29f729425466 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:29.102345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1e78dbeb-ae3d-4d82-8736-29f729425466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:29.438810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 862/1757] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:29.442831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f478f9-fc8b-4ce4-80be-5aa63c810d58 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:29.443166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f478f9-fc8b-4ce4-80be-5aa63c810d58 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:29.443450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f478f9-fc8b-4ce4-80be-5aa63c810d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:29.492049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-34f478f9-fc8b-4ce4-80be-5aa63c810d58 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:29.493341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-34f478f9-fc8b-4ce4-80be-5aa63c810d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:29.524527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 896/1758] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:29.529669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78dbeb-ae3d-4d82-8736-29f729425466 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:29.549901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78dbeb-ae3d-4d82-8736-29f729425466 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:29.566127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:29.567076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:29.567369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:29.567841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:29.568887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:29.569645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:29.572204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:29.604058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 863/1759] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:29.606293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f478f9-fc8b-4ce4-80be-5aa63c810d58 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:29.622082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f478f9-fc8b-4ce4-80be-5aa63c810d58 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:29.638356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa375bca-61d4-4e1b-bf32-15dd4d72fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:29.640724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa375bca-61d4-4e1b-bf32-15dd4d72fec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:29.641481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa375bca-61d4-4e1b-bf32-15dd4d72fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:29.641972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa375bca-61d4-4e1b-bf32-15dd4d72fec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:29.644084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa375bca-61d4-4e1b-bf32-15dd4d72fec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:29.647012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa375bca-61d4-4e1b-bf32-15dd4d72fec7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:29.719043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 897/1760] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:29 2022] POST /identity/v3/projects => generated 399 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:29.723759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:29.741558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e03b52-b220-4731-9a7d-af853a6e824d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:29.960629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4857c3e-6178-45db-889a-ca10ef1c4c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:29.961243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4857c3e-6178-45db-889a-ca10ef1c4c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:29.961561 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4857c3e-6178-45db-889a-ca10ef1c4c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:29.961836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4857c3e-6178-45db-889a-ca10ef1c4c87 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:29.969767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4857c3e-6178-45db-889a-ca10ef1c4c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:29.972774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4857c3e-6178-45db-889a-ca10ef1c4c87 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:30.053911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 864/1761] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:29 2022] POST /identity/v3/users => generated 300 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:30.058550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa375bca-61d4-4e1b-bf32-15dd4d72fec7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:30.078995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa375bca-61d4-4e1b-bf32-15dd4d72fec7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:30.119925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35870db3-fa70-4868-9315-d3b74681c246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:30.122340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35870db3-fa70-4868-9315-d3b74681c246 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:30.124048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35870db3-fa70-4868-9315-d3b74681c246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:30.125455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35870db3-fa70-4868-9315-d3b74681c246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:30.127568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35870db3-fa70-4868-9315-d3b74681c246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:30.132350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35870db3-fa70-4868-9315-d3b74681c246 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:30.179967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 898/1762] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:30.184852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4857c3e-6178-45db-889a-ca10ef1c4c87 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:30.215149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4857c3e-6178-45db-889a-ca10ef1c4c87 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:30.265477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6e38a-e95e-4b1b-98b9-ec8e61f3c7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:30.268215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e38a-e95e-4b1b-98b9-ec8e61f3c7de admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:30.268951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e38a-e95e-4b1b-98b9-ec8e61f3c7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:30.275012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6e38a-e95e-4b1b-98b9-ec8e61f3c7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:30.275795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6e38a-e95e-4b1b-98b9-ec8e61f3c7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:30.277456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6e38a-e95e-4b1b-98b9-ec8e61f3c7de admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:30.335080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 865/1763] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:30.336417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35870db3-fa70-4868-9315-d3b74681c246 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:30.356817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35870db3-fa70-4868-9315-d3b74681c246 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:30.376129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93211a89-0e96-4311-85d8-aba0f5156e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:30.378673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93211a89-0e96-4311-85d8-aba0f5156e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:30.379006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93211a89-0e96-4311-85d8-aba0f5156e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:30.382151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93211a89-0e96-4311-85d8-aba0f5156e58 admin admin] PATH_INFO: `/v3/projects/e20126b6f19547cbb206d4fce06be379/users/f566a1c51cd945e3bd764b9601b42ba7/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:30.454900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93211a89-0e96-4311-85d8-aba0f5156e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20126b6f19547cbb206d4fce06be379, user_id=f566a1c51cd945e3bd764b9601b42ba7, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:30.466372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93211a89-0e96-4311-85d8-aba0f5156e58 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:30.479094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:30.517517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 899/1764] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:30 2022] PUT /identity/v3/projects/e20126b6f19547cbb206d4fce06be379/users/f566a1c51cd945e3bd764b9601b42ba7/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:30.517877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb714de6-eb2d-40d2-91b5-21b775077b26 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:30.518218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb714de6-eb2d-40d2-91b5-21b775077b26 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:30.518711 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb714de6-eb2d-40d2-91b5-21b775077b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:30.644576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fb714de6-eb2d-40d2-91b5-21b775077b26 None None] MFA Rules not processed for user `f566a1c51cd945e3bd764b9601b42ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:30.654452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fb714de6-eb2d-40d2-91b5-21b775077b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:30.671340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:30.888883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93211a89-0e96-4311-85d8-aba0f5156e58 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:30.896815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 866/1765] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:31.141858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93211a89-0e96-4311-85d8-aba0f5156e58 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:31.185469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933238b7-7613-4c58-992d-0417057c7201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:31.186129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933238b7-7613-4c58-992d-0417057c7201 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:31.188340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933238b7-7613-4c58-992d-0417057c7201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:31.188793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933238b7-7613-4c58-992d-0417057c7201 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:31.191834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933238b7-7613-4c58-992d-0417057c7201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:31.199436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933238b7-7613-4c58-992d-0417057c7201 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:31.292536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ded79-b94a-45ce-b4de-5ac253c1b806 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:31.293419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ded79-b94a-45ce-b4de-5ac253c1b806 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:31.293921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ded79-b94a-45ce-b4de-5ac253c1b806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:31.343864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 900/1766] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:30 2022] POST /identity/v3/projects => generated 399 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:31.347587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933238b7-7613-4c58-992d-0417057c7201 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:31.500632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6e2ded79-b94a-45ce-b4de-5ac253c1b806 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:31.503247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6e2ded79-b94a-45ce-b4de-5ac253c1b806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:31.587935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933238b7-7613-4c58-992d-0417057c7201 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:31.605142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e36a63-67c4-4cea-8c18-d6d97c7a6a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:31.605930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36a63-67c4-4cea-8c18-d6d97c7a6a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:31.606434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36a63-67c4-4cea-8c18-d6d97c7a6a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:31.606871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e36a63-67c4-4cea-8c18-d6d97c7a6a94 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:31.607822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e36a63-67c4-4cea-8c18-d6d97c7a6a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:31.610368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e36a63-67c4-4cea-8c18-d6d97c7a6a94 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:31.680771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 901/1767] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:31 2022] POST /identity/v3/users => generated 306 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:31.685520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e36a63-67c4-4cea-8c18-d6d97c7a6a94 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:31.738555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e36a63-67c4-4cea-8c18-d6d97c7a6a94 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:31.760035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352a147-7ab5-4c27-b343-5942823c630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:31.761572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a147-7ab5-4c27-b343-5942823c630a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:31.762413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a147-7ab5-4c27-b343-5942823c630a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:31.763114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a147-7ab5-4c27-b343-5942823c630a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:31.777132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352a147-7ab5-4c27-b343-5942823c630a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:31.782808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352a147-7ab5-4c27-b343-5942823c630a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:31.806679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 867/1768] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:37:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:31.865582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 902/1769] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:37:31.870264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ded79-b94a-45ce-b4de-5ac253c1b806 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:31.894761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ded79-b94a-45ce-b4de-5ac253c1b806 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:31.914234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:31.914488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:31.914640 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:31.914887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] PATH_INFO: `/v3/projects/e667809886f5432c9ff233721dc73ea3/users/9d8a530498264a73935d755b3a443f0f/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:31.992650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e667809886f5432c9ff233721dc73ea3, user_id=9d8a530498264a73935d755b3a443f0f, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:31.998381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:32.022614 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 868/1770] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:31 2022] PUT /identity/v3/projects/e667809886f5432c9ff233721dc73ea3/users/9d8a530498264a73935d755b3a443f0f/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:32.026668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352a147-7ab5-4c27-b343-5942823c630a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:32.041867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352a147-7ab5-4c27-b343-5942823c630a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:32.047800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:32.049697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:32.274571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:32.321840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:32.326328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f6ab1-eaaf-4508-a271-816247ec3854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:32.333852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f6ab1-eaaf-4508-a271-816247ec3854 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:32.335178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f6ab1-eaaf-4508-a271-816247ec3854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:32.336973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f6ab1-eaaf-4508-a271-816247ec3854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:32.346322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f6ab1-eaaf-4508-a271-816247ec3854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:32.348882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f6ab1-eaaf-4508-a271-816247ec3854 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:32.407118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 903/1771] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:32.410316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f6ab1-eaaf-4508-a271-816247ec3854 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:32.465563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f6ab1-eaaf-4508-a271-816247ec3854 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:32.488372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e852bd-5caf-41ec-9438-641f3f99c08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:32.491802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e852bd-5caf-41ec-9438-641f3f99c08d admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:32.492502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e852bd-5caf-41ec-9438-641f3f99c08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:32.494572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e852bd-5caf-41ec-9438-641f3f99c08d admin admin] PATH_INFO: `/v3/projects/e667809886f5432c9ff233721dc73ea3/users/9d8a530498264a73935d755b3a443f0f/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:32.501210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f05eb-9fd3-41ba-ac33-680dc6578f09 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:32.502497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e852bd-5caf-41ec-9438-641f3f99c08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e667809886f5432c9ff233721dc73ea3, user_id=9d8a530498264a73935d755b3a443f0f, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:32.512467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e852bd-5caf-41ec-9438-641f3f99c08d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:32.515059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:32.525782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:32.526920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:32.527745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:32.597393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 904/1772] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:32 2022] PUT /identity/v3/projects/e667809886f5432c9ff233721dc73ea3/users/9d8a530498264a73935d755b3a443f0f/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:32.608629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34de55c-45f2-4b85-8562-d4abb83b843e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:32.609350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34de55c-45f2-4b85-8562-d4abb83b843e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:32.609798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34de55c-45f2-4b85-8562-d4abb83b843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:32.752482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c34de55c-45f2-4b85-8562-d4abb83b843e None None] MFA Rules not processed for user `9d8a530498264a73935d755b3a443f0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:32.757874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c34de55c-45f2-4b85-8562-d4abb83b843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:32.838197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:32.845330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:32.869493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 869/1773] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:37:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 822 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:37:32.884945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:32.887367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.080649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 905/1774] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:32 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:33.133382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319fb59-2eac-4977-a145-eb7f8421ce8e service swift] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:33.280317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb2979-ecec-45f8-9de4-84f907ae53d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:33.282575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2979-ecec-45f8-9de4-84f907ae53d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:33.284045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2979-ecec-45f8-9de4-84f907ae53d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:33.284936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2979-ecec-45f8-9de4-84f907ae53d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:33.290308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-47fb2979-ecec-45f8-9de4-84f907ae53d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.326497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34de55c-45f2-4b85-8562-d4abb83b843e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:33.333186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c34de55c-45f2-4b85-8562-d4abb83b843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:33.363108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34de55c-45f2-4b85-8562-d4abb83b843e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:33.371647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4f580-f1f5-4e0e-a9a0-2f6b60b45956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:33.373084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4f580-f1f5-4e0e-a9a0-2f6b60b45956 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:33.374786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4f580-f1f5-4e0e-a9a0-2f6b60b45956 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:33.375532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4f580-f1f5-4e0e-a9a0-2f6b60b45956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:33.401736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4f580-f1f5-4e0e-a9a0-2f6b60b45956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:33.404061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4f580-f1f5-4e0e-a9a0-2f6b60b45956 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:33.435540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 906/1775] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:33 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:33.516097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:33.565359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb2979-ecec-45f8-9de4-84f907ae53d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:33.568179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb2979-ecec-45f8-9de4-84f907ae53d1 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:33.603166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 870/1776] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:34.154776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4f580-f1f5-4e0e-a9a0-2f6b60b45956 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:34.169101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4f580-f1f5-4e0e-a9a0-2f6b60b45956 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:34.187431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:34.188514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:34.189019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:34.189610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:34.391185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:34.394128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:34.438219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 907/1777] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:35.387892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e54b-9d99-4921-9d2b-f067bd5d6c6f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:35.388520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e54b-9d99-4921-9d2b-f067bd5d6c6f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:35.388696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5e54b-9d99-4921-9d2b-f067bd5d6c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:35.503911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-54c5e54b-9d99-4921-9d2b-f067bd5d6c6f None None] MFA Rules not processed for user `4b46a15d481f4393bef8c4fa2281710b`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:35.508656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-54c5e54b-9d99-4921-9d2b-f067bd5d6c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:35.793725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 871/1778] 10.222.0.49 () {58 vars in 1015 bytes} [Thu Jul 28 06:37:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:36.024101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:36.025829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:36.056731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383f574-00f6-4093-b6da-a6bb46794e6c service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:36.079229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:36.080119 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:36.080589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:36.081061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:36.103044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:36.104815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:36.161670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 908/1779] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:36.606789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8fa74-07c5-403f-ae65-52ff57d177ef None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:36.607364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8fa74-07c5-403f-ae65-52ff57d177ef None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:36.607463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8fa74-07c5-403f-ae65-52ff57d177ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:36.678864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-53d8fa74-07c5-403f-ae65-52ff57d177ef None None] MFA Rules not processed for user `4a1db92e28444854b9de817dd36104ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:36.685025 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-53d8fa74-07c5-403f-ae65-52ff57d177ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:36.889576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 872/1780] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:38.466700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:38.468935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:38.649749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:38.778166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8451-e0b8-44b0-90e7-d1c61bb13272 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:38.801817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:38.804011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:38.805277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:38.808746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] PATH_INFO: `/v3/users/de8001f410884446a391c7d60f7cd02b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:38.812836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8001f410884446a391c7d60f7cd02b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:38.825421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:39.036867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8001f410884446a391c7d60f7cd02b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.072384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8001f410884446a391c7d60f7cd02b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.102272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8001f410884446a391c7d60f7cd02b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.104161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8001f410884446a391c7d60f7cd02b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.187896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8001f410884446a391c7d60f7cd02b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:39.236713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 909/1781] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:38 2022] DELETE /identity/v3/users/de8001f410884446a391c7d60f7cd02b => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:39.241526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98914f-c75b-484e-9bce-1cea1b86a212 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:39.243033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98914f-c75b-484e-9bce-1cea1b86a212 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:39.243706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98914f-c75b-484e-9bce-1cea1b86a212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:39.407453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6e98914f-c75b-484e-9bce-1cea1b86a212 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:39.425417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6e98914f-c75b-484e-9bce-1cea1b86a212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:39.917747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 873/1782] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 679 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:40.094881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:40.099619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:40.417114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e3292-cd9e-4a7a-ab6d-002e2558df4c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:40.458486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:40.475000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:40.478173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:40.478954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:40.554206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:40.557493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:40.581416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 910/1783] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:40.993489 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98914f-c75b-484e-9bce-1cea1b86a212 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:40.995151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6e98914f-c75b-484e-9bce-1cea1b86a212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:41.041467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98914f-c75b-484e-9bce-1cea1b86a212 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:41.075578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:41.081508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:41.081508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:41.081508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] PATH_INFO: `/v3/users/f566a1c51cd945e3bd764b9601b42ba7` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:41.086656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] RBAC: Authorizing `identity:delete_user(user_id=f566a1c51cd945e3bd764b9601b42ba7)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:41.098206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:41.116757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:41.118910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:41.153414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacb5e7-37e2-464c-b0aa-7ce604f140fa service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:41.177061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:41.184155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:41.185785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:41.190080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] PATH_INFO: `/v3/users/b7b5998f6af34e289afb6543a1bc3cc6` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:41.196568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7b5998f6af34e289afb6543a1bc3cc6)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:41.198970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:41.219974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:41.282636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f566a1c51cd945e3bd764b9601b42ba7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.338510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f566a1c51cd945e3bd764b9601b42ba7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.348183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f566a1c51cd945e3bd764b9601b42ba7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.356790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b5998f6af34e289afb6543a1bc3cc6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.391059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:41.395438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b5998f6af34e289afb6543a1bc3cc6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.430380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b5998f6af34e289afb6543a1bc3cc6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.440890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b5998f6af34e289afb6543a1bc3cc6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.441604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f566a1c51cd945e3bd764b9601b42ba7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.443939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0212c74b-b6be-4f20-800a-883927d2dada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f566a1c51cd945e3bd764b9601b42ba7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.499921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 874/1784] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:40 2022] DELETE /identity/v3/users/f566a1c51cd945e3bd764b9601b42ba7 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:41.509054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da471c-2595-44e3-8cef-38d1e4babba5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:41.509665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da471c-2595-44e3-8cef-38d1e4babba5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:41.511082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da471c-2595-44e3-8cef-38d1e4babba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:41.511203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f73fa085-e4ce-4d03-a801-bb15d2038ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b5998f6af34e289afb6543a1bc3cc6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:41.534977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 911/1785] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:41 2022] DELETE /identity/v3/users/b7b5998f6af34e289afb6543a1bc3cc6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:41.541328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9b5e9-dd10-4ccd-9ead-aee77969e0ac None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:41.541755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9b5e9-dd10-4ccd-9ead-aee77969e0ac None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:41.541938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9b5e9-dd10-4ccd-9ead-aee77969e0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:41.595372 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a5da471c-2595-44e3-8cef-38d1e4babba5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:41.601371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-52f9b5e9-dd10-4ccd-9ead-aee77969e0ac None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:41.602925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-52f9b5e9-dd10-4ccd-9ead-aee77969e0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:41.615225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a5da471c-2595-44e3-8cef-38d1e4babba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:41.798147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 912/1786] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:41.818181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 875/1787] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:41.844568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64203c37-0810-488d-afa2-11b3abaef9a8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:41.844866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64203c37-0810-488d-afa2-11b3abaef9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:41.845320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64203c37-0810-488d-afa2-11b3abaef9a8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:41.849392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da471c-2595-44e3-8cef-38d1e4babba5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:41.850916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 913/1788] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:37:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:37:41.851235 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a5da471c-2595-44e3-8cef-38d1e4babba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:41.884356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64203c37-0810-488d-afa2-11b3abaef9a8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:41.886196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-64203c37-0810-488d-afa2-11b3abaef9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:41.959043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da471c-2595-44e3-8cef-38d1e4babba5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:41.979107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c2e73-fd09-476c-b319-7bdac362bfe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:41.979107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2e73-fd09-476c-b319-7bdac362bfe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:41.979107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2e73-fd09-476c-b319-7bdac362bfe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:41.979107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2e73-fd09-476c-b319-7bdac362bfe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:41.995983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c2e73-fd09-476c-b319-7bdac362bfe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:42.000224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c2e73-fd09-476c-b319-7bdac362bfe4 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:42.033069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 876/1789] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:42.035867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fba46-0d32-4481-be4c-8585608aaefa None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:42.036728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fba46-0d32-4481-be4c-8585608aaefa None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:42.045241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fba46-0d32-4481-be4c-8585608aaefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:42.065205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64203c37-0810-488d-afa2-11b3abaef9a8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:42.072808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:42.074022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:42.074505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:42.074983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:42.087167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:42.088964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:42.092250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6b8fba46-0d32-4481-be4c-8585608aaefa None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:42.093679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6b8fba46-0d32-4481-be4c-8585608aaefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:42.102327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 914/1790] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:42.181326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:42.219857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 877/1791] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:37:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:42.246824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:42.249323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:42.282376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef33cd-742c-4cd3-ace9-ae7d8b9f227c service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:42.298440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793f42-2ca6-4e4d-b3b0-1d9adb8ab31a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:42.299101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793f42-2ca6-4e4d-b3b0-1d9adb8ab31a service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:42.299535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793f42-2ca6-4e4d-b3b0-1d9adb8ab31a service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:42.299825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793f42-2ca6-4e4d-b3b0-1d9adb8ab31a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:42.311072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc793f42-2ca6-4e4d-b3b0-1d9adb8ab31a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:42.313180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc793f42-2ca6-4e4d-b3b0-1d9adb8ab31a service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:42.327634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 915/1792] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:42.586782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fba46-0d32-4481-be4c-8585608aaefa None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:42.588947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6b8fba46-0d32-4481-be4c-8585608aaefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:42.637277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fba46-0d32-4481-be4c-8585608aaefa None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:42.645739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:42.646474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:42.647248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:42.647558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] PATH_INFO: `/v3/projects/db906862ca974657a0539df041c7e952` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:42.649048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=db906862ca974657a0539df041c7e952)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:42.651218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:42.762987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db906862ca974657a0539df041c7e952', 'request_id': 'req-b2b6a5b5-918c-44be-8d7f-a039227cfddd'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:42.779048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db906862ca974657a0539df041c7e952', 'request_id': 'req-b2b6a5b5-918c-44be-8d7f-a039227cfddd'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:42.791639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db906862ca974657a0539df041c7e952', 'request_id': 'req-b2b6a5b5-918c-44be-8d7f-a039227cfddd'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:42.807991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] The token cache is being invalidate because project db906862ca974657a0539df041c7e952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:42.808285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:42.811552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:42.818344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 878/1793] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:42 2022] DELETE /identity/v3/projects/db906862ca974657a0539df041c7e952 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:42.822551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793f42-2ca6-4e4d-b3b0-1d9adb8ab31a service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:42.847163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793f42-2ca6-4e4d-b3b0-1d9adb8ab31a service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:42.986432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:42.987754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:42.988110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:42.988960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] PATH_INFO: `/v3/users/1cf9948fc8094fe4832882eeb05eb46c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:42.991919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cf9948fc8094fe4832882eeb05eb46c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:42.998840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:43.003311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:43.095338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cf9948fc8094fe4832882eeb05eb46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.109312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cf9948fc8094fe4832882eeb05eb46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.122693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cf9948fc8094fe4832882eeb05eb46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.123364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cf9948fc8094fe4832882eeb05eb46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.149956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cf9948fc8094fe4832882eeb05eb46c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.167232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 916/1794] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:42 2022] DELETE /identity/v3/users/1cf9948fc8094fe4832882eeb05eb46c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:43.557626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:43.559482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:43.596696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6a5b5-918c-44be-8d7f-a039227cfddd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:43.640603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:43.642926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:43.708854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:43.710908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:43.711436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:43.712355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] PATH_INFO: `/v3/projects/773c6f93b58a42a48700a32a2ce4eb47` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:43.715258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773c6f93b58a42a48700a32a2ce4eb47)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:43.717688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:43.748821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98689a03-312a-4636-9921-7cd1dabf1985 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:43.754004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:43.754870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:43.755491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:43.755900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] PATH_INFO: `/v3/projects/e20126b6f19547cbb206d4fce06be379` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:43.757284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e20126b6f19547cbb206d4fce06be379)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:43.759716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:43.807428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773c6f93b58a42a48700a32a2ce4eb47', 'request_id': 'req-df8cede9-6378-4d10-be9a-c24c77e67e78'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.817427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773c6f93b58a42a48700a32a2ce4eb47', 'request_id': 'req-df8cede9-6378-4d10-be9a-c24c77e67e78'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.825193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773c6f93b58a42a48700a32a2ce4eb47', 'request_id': 'req-df8cede9-6378-4d10-be9a-c24c77e67e78'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.858719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] The token cache is being invalidate because project 773c6f93b58a42a48700a32a2ce4eb47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:43.859375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.861958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-df8cede9-6378-4d10-be9a-c24c77e67e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.865879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e20126b6f19547cbb206d4fce06be379', 'request_id': 'req-26888076-06c9-40f1-a9c2-06c7f2a44809'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.869088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 879/1795] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:43 2022] DELETE /identity/v3/projects/773c6f93b58a42a48700a32a2ce4eb47 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:43.878251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e20126b6f19547cbb206d4fce06be379', 'request_id': 'req-26888076-06c9-40f1-a9c2-06c7f2a44809'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.879696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfb224-607c-47bc-8be9-e0cbb67c01a1 None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:43.880302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfb224-607c-47bc-8be9-e0cbb67c01a1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:43.881461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfb224-607c-47bc-8be9-e0cbb67c01a1 None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:43.889675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 880/1796] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:37:43 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:37:43.891924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e870017-fa75-486a-a01f-ef6ddc8b66cb None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:43.892368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e870017-fa75-486a-a01f-ef6ddc8b66cb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:43.896911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e870017-fa75-486a-a01f-ef6ddc8b66cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:43.900654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e20126b6f19547cbb206d4fce06be379', 'request_id': 'req-26888076-06c9-40f1-a9c2-06c7f2a44809'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.915557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:43.916579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] The token cache is being invalidate because project e20126b6f19547cbb206d4fce06be379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:43.917097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.918843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:43.922123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 917/1797] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:43 2022] DELETE /identity/v3/projects/e20126b6f19547cbb206d4fce06be379 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:43.925530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:43.927825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:43.951455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8e870017-fa75-486a-a01f-ef6ddc8b66cb None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:43.952889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8e870017-fa75-486a-a01f-ef6ddc8b66cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:43.961448 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:44.057351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 881/1798] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:44.059130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3eee8-5ad0-4e52-9a82-429c0734ff63 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:44.059503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3eee8-5ad0-4e52-9a82-429c0734ff63 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:44.060454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3eee8-5ad0-4e52-9a82-429c0734ff63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:44.076962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26888076-06c9-40f1-a9c2-06c7f2a44809 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:44.086146 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:44.088178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:44.089781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:44.089781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] PATH_INFO: `/v3/users/9d8a530498264a73935d755b3a443f0f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:44.091293 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1cf3eee8-5ad0-4e52-9a82-429c0734ff63 None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:44.092420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8a530498264a73935d755b3a443f0f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:44.096449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1cf3eee8-5ad0-4e52-9a82-429c0734ff63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:44.097884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:44.230474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8a530498264a73935d755b3a443f0f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.243972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8a530498264a73935d755b3a443f0f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.255340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8a530498264a73935d755b3a443f0f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.256058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8a530498264a73935d755b3a443f0f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.261091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 882/1799] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:37:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:44.262659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366bca-050e-4c5c-adfe-fed1c2a7bdf8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:44.263009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366bca-050e-4c5c-adfe-fed1c2a7bdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:44.263367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366bca-050e-4c5c-adfe-fed1c2a7bdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:44.297378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8a530498264a73935d755b3a443f0f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.315302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 918/1800] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:43 2022] DELETE /identity/v3/users/9d8a530498264a73935d755b3a443f0f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:44.316278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:44.317923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:44.378161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5a366bca-050e-4c5c-adfe-fed1c2a7bdf8 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:44.386724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5a366bca-050e-4c5c-adfe-fed1c2a7bdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:44.586536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 883/1801] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:44.587684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366bca-050e-4c5c-adfe-fed1c2a7bdf8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:44.589603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5a366bca-050e-4c5c-adfe-fed1c2a7bdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:44.607932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84e288-79c2-499f-bbf3-4e80ec67fbf1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:44.610870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366bca-050e-4c5c-adfe-fed1c2a7bdf8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:44.624619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:44.625317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:44.625682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:44.625988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:44.632324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:44.633779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:44.633914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:44.634375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:44.634980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] PATH_INFO: `/v3/projects/c5dec23576874ca880d454ee34c8feef` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:44.637439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5dec23576874ca880d454ee34c8feef)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:44.648519 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:44.655542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:44.657907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:44.690551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 919/1802] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:44.691998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:44.713470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fe893-0225-4a60-adcf-ffc330fab9c1 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:44.726431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:44.727382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:44.727816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:44.728229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:44.729082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:44.729683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:44.731473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:44.745034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:44.778181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 920/1803] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:44.779709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:44.795566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:44.803712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ba6c-de2d-4b3d-bda2-4b5f0d2cf8e6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:44.824841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5dec23576874ca880d454ee34c8feef', 'request_id': 'req-c8d9633a-7752-4e4b-b844-973aba49e170'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.848492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5dec23576874ca880d454ee34c8feef', 'request_id': 'req-c8d9633a-7752-4e4b-b844-973aba49e170'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.861433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5dec23576874ca880d454ee34c8feef', 'request_id': 'req-c8d9633a-7752-4e4b-b844-973aba49e170'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.902808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] The token cache is being invalidate because project c5dec23576874ca880d454ee34c8feef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:44.903512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.906676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:44.911350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 884/1804] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:44 2022] DELETE /identity/v3/projects/c5dec23576874ca880d454ee34c8feef => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:44.914407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:44.915296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:44.915559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:44.917110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:44.917666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:44.919995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:44.920692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:44.924042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:44.952747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9633a-7752-4e4b-b844-973aba49e170 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:44.969547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:44.970470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:44.970855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:44.971366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] PATH_INFO: `/v3/users/2e0a6262f7774ec2996fa1fa83a10f57` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:44.973339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e0a6262f7774ec2996fa1fa83a10f57)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:44.976160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.000310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 921/1805] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:44 2022] POST /identity/v3/projects => generated 401 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:45.000919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.007235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.069846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e0a6262f7774ec2996fa1fa83a10f57'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.082675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e0a6262f7774ec2996fa1fa83a10f57'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.084450 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e0a6262f7774ec2996fa1fa83a10f57'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.106002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e0a6262f7774ec2996fa1fa83a10f57'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.106894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e0a6262f7774ec2996fa1fa83a10f57'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.124404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 885/1806] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:44 2022] DELETE /identity/v3/users/2e0a6262f7774ec2996fa1fa83a10f57 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:45.125352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.127064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.179746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a77c48-02ec-432c-9699-b1ae67e100c6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.228596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40102b-70ab-4cdf-924a-c56c998dde81 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.237503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.239173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.239569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.239858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] PATH_INFO: `/v3/projects/e667809886f5432c9ff233721dc73ea3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.241454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e667809886f5432c9ff233721dc73ea3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.244170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.275891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.277324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.277784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.278209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.279166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.281622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.311859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:45.316649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 922/1807] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:44 2022] POST /identity/v3/users => generated 301 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:45.321487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.324061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.331114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e667809886f5432c9ff233721dc73ea3', 'request_id': 'req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.341911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228513f-c46e-4615-9455-7610f0fe828c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.345818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e667809886f5432c9ff233721dc73ea3', 'request_id': 'req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.379028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e667809886f5432c9ff233721dc73ea3', 'request_id': 'req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.385989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:45.415049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] The token cache is being invalidate because project e667809886f5432c9ff233721dc73ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:45.415049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.415742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-36cb7c96-da91-453a-97a7-c3a16f9ec8e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.417957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 886/1808] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:45 2022] DELETE /identity/v3/projects/e667809886f5432c9ff233721dc73ea3 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:45.423407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057bf0-9543-4908-9d01-1ceb361dd4eb None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.423900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057bf0-9543-4908-9d01-1ceb361dd4eb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.424555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057bf0-9543-4908-9d01-1ceb361dd4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.446726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.447571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.448005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.448536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.449455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.451213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.465405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 923/1809] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:45.469142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-47057bf0-9543-4908-9d01-1ceb361dd4eb None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:45.469692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.470539 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-47057bf0-9543-4908-9d01-1ceb361dd4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.471685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.487951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 887/1810] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:45.490213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b009020-9552-466a-9e38-83323d4a6a4a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.490641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b009020-9552-466a-9e38-83323d4a6a4a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.491010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b009020-9552-466a-9e38-83323d4a6a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.497774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478b270-5d7f-4a88-8592-174160d7909c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.503553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d824ad-7128-4ba8-8133-6f08c7b713d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.504288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d824ad-7128-4ba8-8133-6f08c7b713d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.504796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d824ad-7128-4ba8-8133-6f08c7b713d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.505172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d824ad-7128-4ba8-8133-6f08c7b713d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.505913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d824ad-7128-4ba8-8133-6f08c7b713d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.507507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d824ad-7128-4ba8-8133-6f08c7b713d4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.511528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7b009020-9552-466a-9e38-83323d4a6a4a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:45.512740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7b009020-9552-466a-9e38-83323d4a6a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.519343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 924/1811] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:37:45.524702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d824ad-7128-4ba8-8133-6f08c7b713d4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.529965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 888/1812] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:45.530993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b009020-9552-466a-9e38-83323d4a6a4a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.538366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d824ad-7128-4ba8-8133-6f08c7b713d4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.545089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175e2ea-52f9-4077-94e9-a119671f11da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.546032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e2ea-52f9-4077-94e9-a119671f11da admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.546414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e2ea-52f9-4077-94e9-a119671f11da admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.546805 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e2ea-52f9-4077-94e9-a119671f11da admin admin] PATH_INFO: `/v3/projects/80ed1e6c1665477fa0858a8ba0c8573a/users/b2da258a601b4acaa289266fad86c8c0/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.547063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b009020-9552-466a-9e38-83323d4a6a4a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.552646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.553218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.553652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.553932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.554631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:45.555120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.556513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.573465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175e2ea-52f9-4077-94e9-a119671f11da admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ed1e6c1665477fa0858a8ba0c8573a, user_id=b2da258a601b4acaa289266fad86c8c0, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.576142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175e2ea-52f9-4077-94e9-a119671f11da admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.579945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 889/1813] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:45.584860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.607505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcc32b-8071-4fe2-ba1a-387f86a828d7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.608900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 925/1814] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:45 2022] PUT /identity/v3/projects/80ed1e6c1665477fa0858a8ba0c8573a/users/b2da258a601b4acaa289266fad86c8c0/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:45.611158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c2cc3-2d9c-422b-bdcf-af3592f70309 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.611619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c2cc3-2d9c-422b-bdcf-af3592f70309 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.612032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c2cc3-2d9c-422b-bdcf-af3592f70309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.656147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4c6c2cc3-2d9c-422b-bdcf-af3592f70309 None None] MFA Rules not processed for user `b2da258a601b4acaa289266fad86c8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:45.662683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4c6c2cc3-2d9c-422b-bdcf-af3592f70309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.678314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.679174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.679704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.680143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.680932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.687210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.733545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 890/1815] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:45 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:45.734771 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.738921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.770539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 926/1816] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:45.774262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c2cc3-2d9c-422b-bdcf-af3592f70309 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.781721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c2cc3-2d9c-422b-bdcf-af3592f70309 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.807311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b39544-b267-4e09-a81a-39a3285678de admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.812270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.812921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.813232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.813665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] PATH_INFO: `/v3/projects/0ac0a495121947bdb152c826e7831839` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.814863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac0a495121947bdb152c826e7831839)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.816832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.839113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754c74-367b-4ebf-8430-986379244cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.839738 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754c74-367b-4ebf-8430-986379244cec admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.840018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754c74-367b-4ebf-8430-986379244cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.840320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754c74-367b-4ebf-8430-986379244cec admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.840988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3754c74-367b-4ebf-8430-986379244cec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.841698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:45.842987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3754c74-367b-4ebf-8430-986379244cec admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.865378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 927/1817] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:45 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:45.866071 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754c74-367b-4ebf-8430-986379244cec admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.872770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:45.875679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac0a495121947bdb152c826e7831839', 'request_id': 'req-ac8516be-2a27-4cf9-b603-895061854ecd'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.876824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754c74-367b-4ebf-8430-986379244cec admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.883813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac0a495121947bdb152c826e7831839', 'request_id': 'req-ac8516be-2a27-4cf9-b603-895061854ecd'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.889922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac0a495121947bdb152c826e7831839', 'request_id': 'req-ac8516be-2a27-4cf9-b603-895061854ecd'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.902788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] The token cache is being invalidate because project 0ac0a495121947bdb152c826e7831839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:45.903071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.903737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:45.905277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 891/1818] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:45 2022] DELETE /identity/v3/projects/0ac0a495121947bdb152c826e7831839 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:45.906205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:45.907501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:45.937174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db1893-a0d0-4ad0-86c1-dce291b216a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:45.937832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db1893-a0d0-4ad0-86c1-dce291b216a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.938117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db1893-a0d0-4ad0-86c1-dce291b216a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.938429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db1893-a0d0-4ad0-86c1-dce291b216a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.939101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db1893-a0d0-4ad0-86c1-dce291b216a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:45.941088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db1893-a0d0-4ad0-86c1-dce291b216a8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:45.969013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 928/1819] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:45 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:45.970319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70756d0b-59c1-43c4-9617-5d2a9b6d166b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:45.970608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70756d0b-59c1-43c4-9617-5d2a9b6d166b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:45.971012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70756d0b-59c1-43c4-9617-5d2a9b6d166b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:45.973512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8516be-2a27-4cf9-b603-895061854ecd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:45.991027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-70756d0b-59c1-43c4-9617-5d2a9b6d166b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:45.992031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-70756d0b-59c1-43c4-9617-5d2a9b6d166b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.032244 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.032908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.033216 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.033525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.034229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.036557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.046477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 892/1820] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:46.047266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.048934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.056516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 929/1821] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:46.057539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70756d0b-59c1-43c4-9617-5d2a9b6d166b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.067191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54994a3-96a9-48b1-9924-525a89206400 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.067460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70756d0b-59c1-43c4-9617-5d2a9b6d166b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.072104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2036b-851b-4454-8fa7-3e4b6f2c616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.072234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8548218-641c-466b-bb6c-6cc8cfc21ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.072715 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2036b-851b-4454-8fa7-3e4b6f2c616f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.072917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8548218-641c-466b-bb6c-6cc8cfc21ddc admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.073019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2036b-851b-4454-8fa7-3e4b6f2c616f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.073267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2036b-851b-4454-8fa7-3e4b6f2c616f admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.073406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8548218-641c-466b-bb6c-6cc8cfc21ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.073749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8548218-641c-466b-bb6c-6cc8cfc21ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.073981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2036b-851b-4454-8fa7-3e4b6f2c616f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.074571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8548218-641c-466b-bb6c-6cc8cfc21ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.075872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2036b-851b-4454-8fa7-3e4b6f2c616f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.076258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8548218-641c-466b-bb6c-6cc8cfc21ddc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.087178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 930/1822] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:46.088534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391203d-0bb0-493e-b288-e365556b93e6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.088822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391203d-0bb0-493e-b288-e365556b93e6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.089117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391203d-0bb0-493e-b288-e365556b93e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.102083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 893/1823] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/users => generated 308 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:46.102083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2036b-851b-4454-8fa7-3e4b6f2c616f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.109265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8391203d-0bb0-493e-b288-e365556b93e6 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:46.109534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2036b-851b-4454-8fa7-3e4b6f2c616f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.110485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8391203d-0bb0-493e-b288-e365556b93e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.114731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd5a77-41a1-4340-9e1c-dd4190876fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.115518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5a77-41a1-4340-9e1c-dd4190876fed admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.115821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5a77-41a1-4340-9e1c-dd4190876fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.116085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5a77-41a1-4340-9e1c-dd4190876fed admin admin] PATH_INFO: `/v3/projects/195dd8554c7743288381d115015351b5/users/6f85cf8b029b4b579570e0d8d5c27926/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.121600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 931/1824] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:46.121600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391203d-0bb0-493e-b288-e365556b93e6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.128933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391203d-0bb0-493e-b288-e365556b93e6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.131484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd5a77-41a1-4340-9e1c-dd4190876fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=195dd8554c7743288381d115015351b5, user_id=6f85cf8b029b4b579570e0d8d5c27926, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.133697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b45ac4-c3bb-449d-a625-4e544d8caa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.134170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd5a77-41a1-4340-9e1c-dd4190876fed admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.134294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b45ac4-c3bb-449d-a625-4e544d8caa60 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.134580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b45ac4-c3bb-449d-a625-4e544d8caa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.134852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b45ac4-c3bb-449d-a625-4e544d8caa60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.135675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b45ac4-c3bb-449d-a625-4e544d8caa60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.137235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b45ac4-c3bb-449d-a625-4e544d8caa60 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.147926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 932/1825] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:46.151743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 894/1826] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:46 2022] PUT /identity/v3/projects/195dd8554c7743288381d115015351b5/users/6f85cf8b029b4b579570e0d8d5c27926/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:46.151969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b45ac4-c3bb-449d-a625-4e544d8caa60 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.152370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd5a77-41a1-4340-9e1c-dd4190876fed admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.158873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b45ac4-c3bb-449d-a625-4e544d8caa60 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.165862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:46.221111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9a537-b87e-44e0-ac3a-5635122d3269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.221765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9a537-b87e-44e0-ac3a-5635122d3269 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.222053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9a537-b87e-44e0-ac3a-5635122d3269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.222330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9a537-b87e-44e0-ac3a-5635122d3269 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.223862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9a537-b87e-44e0-ac3a-5635122d3269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:46.224177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9a537-b87e-44e0-ac3a-5635122d3269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.226009 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9a537-b87e-44e0-ac3a-5635122d3269 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.229676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:46.232098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd5a77-41a1-4340-9e1c-dd4190876fed admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.241061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51592e84-b59a-4727-a4b4-07b516cb77ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.241788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51592e84-b59a-4727-a4b4-07b516cb77ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.242075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51592e84-b59a-4727-a4b4-07b516cb77ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.242607 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51592e84-b59a-4727-a4b4-07b516cb77ec admin admin] PATH_INFO: `/v3/projects/e57753859d184a50b3c818952730eebf/users/2b1b4f43e17b4406851c15e5ecffcd56/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.242724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 933/1827] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:46.243892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd01c0-cbf8-4470-8ddb-66c5ac5ac9a9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.244240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd01c0-cbf8-4470-8ddb-66c5ac5ac9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.244528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd01c0-cbf8-4470-8ddb-66c5ac5ac9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.265790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5bfd01c0-cbf8-4470-8ddb-66c5ac5ac9a9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:46.266022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51592e84-b59a-4727-a4b4-07b516cb77ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57753859d184a50b3c818952730eebf, user_id=2b1b4f43e17b4406851c15e5ecffcd56, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.267597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5bfd01c0-cbf8-4470-8ddb-66c5ac5ac9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.268665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51592e84-b59a-4727-a4b4-07b516cb77ec admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.282016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 934/1828] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:46.282762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd01c0-cbf8-4470-8ddb-66c5ac5ac9a9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.284433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 895/1829] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:46 2022] PUT /identity/v3/projects/e57753859d184a50b3c818952730eebf/users/2b1b4f43e17b4406851c15e5ecffcd56/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:46.288819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51592e84-b59a-4727-a4b4-07b516cb77ec admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.294672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd01c0-cbf8-4470-8ddb-66c5ac5ac9a9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.360480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.360480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.360480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.360480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.360876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.362240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.371016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51592e84-b59a-4727-a4b4-07b516cb77ec admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.376530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d0087-5767-4829-9860-e62d1cc57d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.377188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0087-5767-4829-9860-e62d1cc57d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.377507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0087-5767-4829-9860-e62d1cc57d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.377781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0087-5767-4829-9860-e62d1cc57d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.378510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d0087-5767-4829-9860-e62d1cc57d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.381035 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d0087-5767-4829-9860-e62d1cc57d86 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.395957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 935/1830] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:46.397454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.398844 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 896/1831] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:37:46.400459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.400919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d0087-5767-4829-9860-e62d1cc57d86 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.411799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d0087-5767-4829-9860-e62d1cc57d86 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.466089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49529e-6e93-4b6e-88f5-e2d5d8281a19 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.469681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7b9dd-5519-4809-8bd9-db6f483febc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.470300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7b9dd-5519-4809-8bd9-db6f483febc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.470580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7b9dd-5519-4809-8bd9-db6f483febc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.470756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f894a-893e-4fe2-aafc-f9d863b46393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.470872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7b9dd-5519-4809-8bd9-db6f483febc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.471388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f894a-893e-4fe2-aafc-f9d863b46393 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.471671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f894a-893e-4fe2-aafc-f9d863b46393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.471945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f894a-893e-4fe2-aafc-f9d863b46393 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.472713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f894a-893e-4fe2-aafc-f9d863b46393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.474687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f894a-893e-4fe2-aafc-f9d863b46393 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.480186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7b9dd-5519-4809-8bd9-db6f483febc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.481712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7b9dd-5519-4809-8bd9-db6f483febc7 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.492076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 936/1832] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:46.492886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7b9dd-5519-4809-8bd9-db6f483febc7 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.497355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 897/1833] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:46.504862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f894a-893e-4fe2-aafc-f9d863b46393 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.508030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7b9dd-5519-4809-8bd9-db6f483febc7 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.513985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053dc83-5630-4cfc-ad8a-13ee4ee2172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.514716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053dc83-5630-4cfc-ad8a-13ee4ee2172c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.515036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053dc83-5630-4cfc-ad8a-13ee4ee2172c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.515143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f894a-893e-4fe2-aafc-f9d863b46393 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.515305 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053dc83-5630-4cfc-ad8a-13ee4ee2172c admin admin] PATH_INFO: `/v3/projects/e57753859d184a50b3c818952730eebf/users/2b1b4f43e17b4406851c15e5ecffcd56/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.517380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053dc83-5630-4cfc-ad8a-13ee4ee2172c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57753859d184a50b3c818952730eebf, user_id=2b1b4f43e17b4406851c15e5ecffcd56, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.520807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053dc83-5630-4cfc-ad8a-13ee4ee2172c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.521265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67aa43-94fb-41b1-89c2-800d644b5f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.521967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67aa43-94fb-41b1-89c2-800d644b5f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.522251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67aa43-94fb-41b1-89c2-800d644b5f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.522531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67aa43-94fb-41b1-89c2-800d644b5f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.523260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67aa43-94fb-41b1-89c2-800d644b5f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.525024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67aa43-94fb-41b1-89c2-800d644b5f80 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.532382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 937/1834] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:46 2022] PUT /identity/v3/projects/e57753859d184a50b3c818952730eebf/users/2b1b4f43e17b4406851c15e5ecffcd56/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:46.541126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 898/1835] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:37:46.543820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053dc83-5630-4cfc-ad8a-13ee4ee2172c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.552378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dad496-d887-4dc7-beb2-8b6caf415e50 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.553818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dad496-d887-4dc7-beb2-8b6caf415e50 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.555060 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dad496-d887-4dc7-beb2-8b6caf415e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.580327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:46.593470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a2dad496-d887-4dc7-beb2-8b6caf415e50 None None] MFA Rules not processed for user `2b1b4f43e17b4406851c15e5ecffcd56`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:46.596260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a2dad496-d887-4dc7-beb2-8b6caf415e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.654747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053dc83-5630-4cfc-ad8a-13ee4ee2172c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.661047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7579a9-13b7-436a-ad4b-30c7c174e779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.664628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7579a9-13b7-436a-ad4b-30c7c174e779 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.664915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7579a9-13b7-436a-ad4b-30c7c174e779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.665194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7579a9-13b7-436a-ad4b-30c7c174e779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.666243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7579a9-13b7-436a-ad4b-30c7c174e779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.668223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7579a9-13b7-436a-ad4b-30c7c174e779 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.675181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:46.684905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 938/1836] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:46.686184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7579a9-13b7-436a-ad4b-30c7c174e779 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.697852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7579a9-13b7-436a-ad4b-30c7c174e779 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.700514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 899/1837] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:46.701583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dad496-d887-4dc7-beb2-8b6caf415e50 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.703148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a2dad496-d887-4dc7-beb2-8b6caf415e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.704601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389f643-5900-40a7-9a26-03d5652e68c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.705930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f643-5900-40a7-9a26-03d5652e68c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.706393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f643-5900-40a7-9a26-03d5652e68c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.707681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f643-5900-40a7-9a26-03d5652e68c9 admin admin] PATH_INFO: `/v3/projects/850b9d5ad3104cc38fae72462130ca9d/users/afb328ad83c84f728cd87e51c77aeb2c/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.731124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389f643-5900-40a7-9a26-03d5652e68c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=850b9d5ad3104cc38fae72462130ca9d, user_id=afb328ad83c84f728cd87e51c77aeb2c, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.734067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389f643-5900-40a7-9a26-03d5652e68c9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.751352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 939/1838] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:46 2022] PUT /identity/v3/projects/850b9d5ad3104cc38fae72462130ca9d/users/afb328ad83c84f728cd87e51c77aeb2c/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:46.764478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bae979-5482-446e-8a29-5ac3cb9534df None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.764478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bae979-5482-446e-8a29-5ac3cb9534df None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.764796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bae979-5482-446e-8a29-5ac3cb9534df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.792069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dad496-d887-4dc7-beb2-8b6caf415e50 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:46.795472 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-37bae979-5482-446e-8a29-5ac3cb9534df None None] MFA Rules not processed for user `afb328ad83c84f728cd87e51c77aeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:46.797480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-37bae979-5482-446e-8a29-5ac3cb9534df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:46.873200 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983be8d-8e0e-46e0-a659-3fc526dac036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:46.879300 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983be8d-8e0e-46e0-a659-3fc526dac036 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:46.880335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983be8d-8e0e-46e0-a659-3fc526dac036 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:46.881025 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983be8d-8e0e-46e0-a659-3fc526dac036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:46.891418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 940/1839] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:46.892476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bae979-5482-446e-8a29-5ac3cb9534df None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:46.976208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983be8d-8e0e-46e0-a659-3fc526dac036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:46.977814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983be8d-8e0e-46e0-a659-3fc526dac036 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:46.989784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:46.990267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 900/1840] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:46 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:46.997536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bae979-5482-446e-8a29-5ac3cb9534df None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.004130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a936-a090-4028-9c13-a65acde97c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.004130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a936-a090-4028-9c13-a65acde97c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.004611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a936-a090-4028-9c13-a65acde97c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.004982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a936-a090-4028-9c13-a65acde97c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.005522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9a936-a090-4028-9c13-a65acde97c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.007436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9a936-a090-4028-9c13-a65acde97c9d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.053355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 941/1841] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:46 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:47.056507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983be8d-8e0e-46e0-a659-3fc526dac036 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.075175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983be8d-8e0e-46e0-a659-3fc526dac036 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.146749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c381d-047f-4e1d-befc-bedcc6f25af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.147371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c381d-047f-4e1d-befc-bedcc6f25af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.147655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c381d-047f-4e1d-befc-bedcc6f25af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.147961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c381d-047f-4e1d-befc-bedcc6f25af2 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.149575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c381d-047f-4e1d-befc-bedcc6f25af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.151498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c381d-047f-4e1d-befc-bedcc6f25af2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.174422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 901/1842] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:47.179152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a936-a090-4028-9c13-a65acde97c9d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.189745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a936-a090-4028-9c13-a65acde97c9d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.195186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54dc69-60b9-4aea-ae32-6672b2ab03aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.195800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54dc69-60b9-4aea-ae32-6672b2ab03aa admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.196080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54dc69-60b9-4aea-ae32-6672b2ab03aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.196353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54dc69-60b9-4aea-ae32-6672b2ab03aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.197057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54dc69-60b9-4aea-ae32-6672b2ab03aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.201648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54dc69-60b9-4aea-ae32-6672b2ab03aa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.212962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 942/1843] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:37:47.217256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c381d-047f-4e1d-befc-bedcc6f25af2 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.228389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c381d-047f-4e1d-befc-bedcc6f25af2 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.233331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ccfec-66a0-4181-b582-f1d243e7b38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.234026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ccfec-66a0-4181-b582-f1d243e7b38c admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.234305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ccfec-66a0-4181-b582-f1d243e7b38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.234578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ccfec-66a0-4181-b582-f1d243e7b38c admin admin] PATH_INFO: `/v3/projects/7f40e4ffa6844b81bbaedd5384ba6b57/users/20b4f2e1b6e74712bd6b97194030bff8/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.250892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ccfec-66a0-4181-b582-f1d243e7b38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f40e4ffa6844b81bbaedd5384ba6b57, user_id=20b4f2e1b6e74712bd6b97194030bff8, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.253683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ccfec-66a0-4181-b582-f1d243e7b38c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.268745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 902/1844] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:47 2022] PUT /identity/v3/projects/7f40e4ffa6844b81bbaedd5384ba6b57/users/20b4f2e1b6e74712bd6b97194030bff8/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:47.272758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54dc69-60b9-4aea-ae32-6672b2ab03aa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.283222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54dc69-60b9-4aea-ae32-6672b2ab03aa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.324625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:47.391148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ccfec-66a0-4181-b582-f1d243e7b38c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.402577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf8b96-41d8-4aa8-9bbd-7b617b21e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.403236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8b96-41d8-4aa8-9bbd-7b617b21e85d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.403584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8b96-41d8-4aa8-9bbd-7b617b21e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.403816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8b96-41d8-4aa8-9bbd-7b617b21e85d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.404479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf8b96-41d8-4aa8-9bbd-7b617b21e85d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.406138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf8b96-41d8-4aa8-9bbd-7b617b21e85d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.407777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ccfec-66a0-4181-b582-f1d243e7b38c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.424840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 943/1845] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:47.424990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.424990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.424990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.425276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] PATH_INFO: `/v3/users/b2da258a601b4acaa289266fad86c8c0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.426934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf8b96-41d8-4aa8-9bbd-7b617b21e85d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:47.428499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2da258a601b4acaa289266fad86c8c0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.441941 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf8b96-41d8-4aa8-9bbd-7b617b21e85d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:47.442191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.448010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80b2bf-8144-400d-af48-4600cbf04914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:47.449018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80b2bf-8144-400d-af48-4600cbf04914 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.449329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80b2bf-8144-400d-af48-4600cbf04914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.449689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80b2bf-8144-400d-af48-4600cbf04914 admin admin] PATH_INFO: `/v3/projects/7f40e4ffa6844b81bbaedd5384ba6b57/users/20b4f2e1b6e74712bd6b97194030bff8/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.451722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80b2bf-8144-400d-af48-4600cbf04914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f40e4ffa6844b81bbaedd5384ba6b57, user_id=20b4f2e1b6e74712bd6b97194030bff8, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:47.455918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80b2bf-8144-400d-af48-4600cbf04914 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:47.477348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 944/1846] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:47 2022] PUT /identity/v3/projects/7f40e4ffa6844b81bbaedd5384ba6b57/users/20b4f2e1b6e74712bd6b97194030bff8/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:47.477348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93679f-97eb-43c2-9729-dc23fb796e6e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.477348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93679f-97eb-43c2-9729-dc23fb796e6e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.477348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93679f-97eb-43c2-9729-dc23fb796e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.559972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2da258a601b4acaa289266fad86c8c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.562364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ed93679f-97eb-43c2-9729-dc23fb796e6e None None] MFA Rules not processed for user `6f85cf8b029b4b579570e0d8d5c27926`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:47.563828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ed93679f-97eb-43c2-9729-dc23fb796e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:47.574264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2da258a601b4acaa289266fad86c8c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.575189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2da258a601b4acaa289266fad86c8c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.631557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2da258a601b4acaa289266fad86c8c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.638423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b151b066-4e74-4538-94f4-d72693f088be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2da258a601b4acaa289266fad86c8c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:47.688695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 903/1847] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:47 2022] DELETE /identity/v3/users/b2da258a601b4acaa289266fad86c8c0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:47.690285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd83b7-91f6-4843-ade9-c5f54b1b9691 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.690554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd83b7-91f6-4843-ade9-c5f54b1b9691 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.690727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd83b7-91f6-4843-ade9-c5f54b1b9691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.782299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bfcd83b7-91f6-4843-ade9-c5f54b1b9691 None None] MFA Rules not processed for user `20b4f2e1b6e74712bd6b97194030bff8`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:47.783517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bfcd83b7-91f6-4843-ade9-c5f54b1b9691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:47.794244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 945/1848] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:47.796113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a65d4-ca15-4801-95fb-5d0efadbe5d4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:47.796484 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a65d4-ca15-4801-95fb-5d0efadbe5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:47.796890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a65d4-ca15-4801-95fb-5d0efadbe5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:47.826117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c46a65d4-ca15-4801-95fb-5d0efadbe5d4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:47.827877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c46a65d4-ca15-4801-95fb-5d0efadbe5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:47.851538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:48.012681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 946/1849] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:48.013924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a65d4-ca15-4801-95fb-5d0efadbe5d4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.015707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c46a65d4-ca15-4801-95fb-5d0efadbe5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:48.031040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a65d4-ca15-4801-95fb-5d0efadbe5d4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.037485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.038416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.039027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.039460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.040297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.042429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.063475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 904/1850] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:48.064181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd83b7-91f6-4843-ade9-c5f54b1b9691 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.066787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bfcd83b7-91f6-4843-ade9-c5f54b1b9691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:48.103594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 947/1851] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:48 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:48.104488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.116674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:48.182670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:48.201131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd83b7-91f6-4843-ade9-c5f54b1b9691 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.225663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34af401-535d-437f-b844-4274dbc15a89 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.231737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c4283-4f47-4e6f-aec6-d735a083f555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.232560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c4283-4f47-4e6f-aec6-d735a083f555 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.232823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c4283-4f47-4e6f-aec6-d735a083f555 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.233358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c4283-4f47-4e6f-aec6-d735a083f555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.317342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192458b4-6251-4dc2-a44f-ac26adb21030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.319052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192458b4-6251-4dc2-a44f-ac26adb21030 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.320233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192458b4-6251-4dc2-a44f-ac26adb21030 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.321038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192458b4-6251-4dc2-a44f-ac26adb21030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.384746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c4283-4f47-4e6f-aec6-d735a083f555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.386610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c4283-4f47-4e6f-aec6-d735a083f555 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.412107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 948/1852] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:48.413056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c4283-4f47-4e6f-aec6-d735a083f555 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.505217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192458b4-6251-4dc2-a44f-ac26adb21030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.507297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192458b4-6251-4dc2-a44f-ac26adb21030 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.527585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 905/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:48.544159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192458b4-6251-4dc2-a44f-ac26adb21030 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.554383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c4283-4f47-4e6f-aec6-d735a083f555 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.565320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192458b4-6251-4dc2-a44f-ac26adb21030 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.574159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce79181-433c-4aab-a35b-17cda6cc55da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.574159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79181-433c-4aab-a35b-17cda6cc55da admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.574504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79181-433c-4aab-a35b-17cda6cc55da admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.574842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79181-433c-4aab-a35b-17cda6cc55da admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.584178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce79181-433c-4aab-a35b-17cda6cc55da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.586315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce79181-433c-4aab-a35b-17cda6cc55da admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.596232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.596751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.597038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.597337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.599471 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:48.625602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 949/1854] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:48 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:48.626715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce79181-433c-4aab-a35b-17cda6cc55da admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.636804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:48.638981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce79181-433c-4aab-a35b-17cda6cc55da admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.647186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29c7e5-403f-4254-a4db-9ff3b025f173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.648251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29c7e5-403f-4254-a4db-9ff3b025f173 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.648574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29c7e5-403f-4254-a4db-9ff3b025f173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.648950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29c7e5-403f-4254-a4db-9ff3b025f173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.649749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29c7e5-403f-4254-a4db-9ff3b025f173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.651610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29c7e5-403f-4254-a4db-9ff3b025f173 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.664427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 950/1855] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:48.668755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29c7e5-403f-4254-a4db-9ff3b025f173 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.680627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29c7e5-403f-4254-a4db-9ff3b025f173 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.684193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.686180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.693121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47691bc4-5e1b-4b0e-b188-c77f234a7b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.694106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691bc4-5e1b-4b0e-b188-c77f234a7b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.694648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691bc4-5e1b-4b0e-b188-c77f234a7b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.695084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691bc4-5e1b-4b0e-b188-c77f234a7b91 admin admin] PATH_INFO: `/v3/projects/57a8c87f22814acb8bdff213e3267da0/users/16e1c92765854bb798963302db0d768d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.709669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 906/1856] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:48.715456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47691bc4-5e1b-4b0e-b188-c77f234a7b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a8c87f22814acb8bdff213e3267da0, user_id=16e1c92765854bb798963302db0d768d, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.718199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47691bc4-5e1b-4b0e-b188-c77f234a7b91 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.743243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 951/1857] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:48 2022] PUT /identity/v3/projects/57a8c87f22814acb8bdff213e3267da0/users/16e1c92765854bb798963302db0d768d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:48.744527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.760324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a979e53-0082-4d29-9208-b38f128603b5 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.906489 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.907186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.907508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.907866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.908655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.911234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:48.923620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 907/1858] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:48.928455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47691bc4-5e1b-4b0e-b188-c77f234a7b91 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:48.945364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47691bc4-5e1b-4b0e-b188-c77f234a7b91 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:48.958265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:48.959094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:48.959813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:48.960174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] PATH_INFO: `/v3/projects/57a8c87f22814acb8bdff213e3267da0/users/16e1c92765854bb798963302db0d768d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:48.962892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a8c87f22814acb8bdff213e3267da0, user_id=16e1c92765854bb798963302db0d768d, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:48.967934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:49.007912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 952/1859] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:48 2022] PUT /identity/v3/projects/57a8c87f22814acb8bdff213e3267da0/users/16e1c92765854bb798963302db0d768d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:49.089769 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:49.094796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:49.111553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2224e-595b-4c08-9674-fbd2e09fb9e9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:49.266247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:49.266928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:49.267210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:49.267500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] PATH_INFO: `/v3/projects/80ed1e6c1665477fa0858a8ba0c8573a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:49.268706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ed1e6c1665477fa0858a8ba0c8573a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:49.271307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:49.392355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ed1e6c1665477fa0858a8ba0c8573a', 'request_id': 'req-39ddc6a5-b841-4cf0-b660-db0d74925111'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.413160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ed1e6c1665477fa0858a8ba0c8573a', 'request_id': 'req-39ddc6a5-b841-4cf0-b660-db0d74925111'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.428937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ed1e6c1665477fa0858a8ba0c8573a', 'request_id': 'req-39ddc6a5-b841-4cf0-b660-db0d74925111'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.430363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:49.450459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] The token cache is being invalidate because project 80ed1e6c1665477fa0858a8ba0c8573a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:49.451013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.456996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-39ddc6a5-b841-4cf0-b660-db0d74925111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.463699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 908/1860] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:49 2022] DELETE /identity/v3/projects/80ed1e6c1665477fa0858a8ba0c8573a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:49.464656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:49.466512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:49.491670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b705da9-6fcb-4337-b822-87a983c6d3b7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:49.542665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:49.645598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:49.648265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:49.648795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:49.649324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] PATH_INFO: `/v3/users/2b1b4f43e17b4406851c15e5ecffcd56` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:49.655587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1b4f43e17b4406851c15e5ecffcd56)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:49.659307 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:49.849236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1b4f43e17b4406851c15e5ecffcd56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.881602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1b4f43e17b4406851c15e5ecffcd56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.923489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1b4f43e17b4406851c15e5ecffcd56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.924443 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1b4f43e17b4406851c15e5ecffcd56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:49.983365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1b4f43e17b4406851c15e5ecffcd56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:50.002129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 953/1861] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:49 2022] DELETE /identity/v3/users/2b1b4f43e17b4406851c15e5ecffcd56 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:51.176621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9ef3d-2631-4a1d-a474-0caf374d8140 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:51.177187 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9ef3d-2631-4a1d-a474-0caf374d8140 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:51.177187 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9ef3d-2631-4a1d-a474-0caf374d8140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:51.222979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-83b9ef3d-2631-4a1d-a474-0caf374d8140 None None] MFA Rules not processed for user `16e1c92765854bb798963302db0d768d`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:51.224518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-83b9ef3d-2631-4a1d-a474-0caf374d8140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:51.340347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:51.342495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:51.373064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 909/1862] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:51.382166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9ef3d-2631-4a1d-a474-0caf374d8140 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:51.383728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-83b9ef3d-2631-4a1d-a474-0caf374d8140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:51.455898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ec395-3c0b-41d8-9be4-4eb7ad8f8481 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:51.470703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:51.471388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:51.471678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:51.471959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] PATH_INFO: `/v3/projects/e57753859d184a50b3c818952730eebf` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:51.474379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57753859d184a50b3c818952730eebf)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:51.476560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:51.505962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9ef3d-2631-4a1d-a474-0caf374d8140 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:51.519045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244443c-dcb2-4ae5-9ff5-d3504dc5a3a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:51.520825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244443c-dcb2-4ae5-9ff5-d3504dc5a3a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:51.522834 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244443c-dcb2-4ae5-9ff5-d3504dc5a3a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:51.523290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244443c-dcb2-4ae5-9ff5-d3504dc5a3a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:51.527720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e244443c-dcb2-4ae5-9ff5-d3504dc5a3a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:51.570275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:51.589546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57753859d184a50b3c818952730eebf', 'request_id': 'req-6481b199-81de-4707-8563-03d061fb4034'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:51.627978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57753859d184a50b3c818952730eebf', 'request_id': 'req-6481b199-81de-4707-8563-03d061fb4034'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:51.640148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244443c-dcb2-4ae5-9ff5-d3504dc5a3a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:51.642658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244443c-dcb2-4ae5-9ff5-d3504dc5a3a2 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:51.653656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57753859d184a50b3c818952730eebf', 'request_id': 'req-6481b199-81de-4707-8563-03d061fb4034'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:51.662940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 910/1863] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:51.688845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] The token cache is being invalidate because project e57753859d184a50b3c818952730eebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:51.689078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:51.689809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6481b199-81de-4707-8563-03d061fb4034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:51.696377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 954/1864] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:51 2022] DELETE /identity/v3/projects/e57753859d184a50b3c818952730eebf => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:51.699055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c058-7ced-4c05-983d-07921083c4d0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:51.699354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c058-7ced-4c05-983d-07921083c4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:51.699633 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c058-7ced-4c05-983d-07921083c4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:51.746766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2197c058-7ced-4c05-983d-07921083c4d0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:51.748183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2197c058-7ced-4c05-983d-07921083c4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:51.793811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:51.844958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 911/1865] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:51.854479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba94145-f273-463f-aa33-819c8b8d82f5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:51.855067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba94145-f273-463f-aa33-819c8b8d82f5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:51.855195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba94145-f273-463f-aa33-819c8b8d82f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:51.891145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6ba94145-f273-463f-aa33-819c8b8d82f5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:51.892599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ba94145-f273-463f-aa33-819c8b8d82f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:51.905904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 955/1866] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:51.911240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197c058-7ced-4c05-983d-07921083c4d0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:51.926195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197c058-7ced-4c05-983d-07921083c4d0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:51.933283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:51.934241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:51.935604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:51.935970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:51.936867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:51.937251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:51.940251 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:51.962740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 912/1867] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:51.966672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba94145-f273-463f-aa33-819c8b8d82f5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:51.979809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba94145-f273-463f-aa33-819c8b8d82f5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:51.985121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305d6e7-ee49-4ee0-ac70-a9f45debfb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:51.985791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305d6e7-ee49-4ee0-ac70-a9f45debfb44 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:51.986063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305d6e7-ee49-4ee0-ac70-a9f45debfb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:51.986335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305d6e7-ee49-4ee0-ac70-a9f45debfb44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:51.987176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305d6e7-ee49-4ee0-ac70-a9f45debfb44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:51.990453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305d6e7-ee49-4ee0-ac70-a9f45debfb44 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:52.071845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 956/1868] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:51 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:52.076132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:52.089165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43820bbf-2a52-41e4-9abd-68bee5408b0d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:52.257587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8d0d7-3b4a-4ecf-9875-bc88329ca94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:52.259576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8d0d7-3b4a-4ecf-9875-bc88329ca94e admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:52.259866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8d0d7-3b4a-4ecf-9875-bc88329ca94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:52.260218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8d0d7-3b4a-4ecf-9875-bc88329ca94e admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:52.262073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8d0d7-3b4a-4ecf-9875-bc88329ca94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:52.286089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8d0d7-3b4a-4ecf-9875-bc88329ca94e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:52.354195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305d6e7-ee49-4ee0-ac70-a9f45debfb44 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:52.354909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 913/1869] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:52 2022] POST /identity/v3/users => generated 290 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:52.367244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305d6e7-ee49-4ee0-ac70-a9f45debfb44 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:52.380659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a166-2e93-4fae-b625-f29ee7ac5b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:52.381472 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a166-2e93-4fae-b625-f29ee7ac5b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:52.381828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a166-2e93-4fae-b625-f29ee7ac5b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:52.382175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a166-2e93-4fae-b625-f29ee7ac5b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:52.382951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4a166-2e93-4fae-b625-f29ee7ac5b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:52.390334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4a166-2e93-4fae-b625-f29ee7ac5b93 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:52.404341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 957/1870] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:52.408554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8d0d7-3b4a-4ecf-9875-bc88329ca94e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:52.432962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8d0d7-3b4a-4ecf-9875-bc88329ca94e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:52.440892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5d824-8bbb-4c7e-99b8-d553531375cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:52.442118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5d824-8bbb-4c7e-99b8-d553531375cf admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:52.442410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5d824-8bbb-4c7e-99b8-d553531375cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:52.442690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5d824-8bbb-4c7e-99b8-d553531375cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:52.443398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5d824-8bbb-4c7e-99b8-d553531375cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:52.446141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5d824-8bbb-4c7e-99b8-d553531375cf admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:52.471191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a166-2e93-4fae-b625-f29ee7ac5b93 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:52.471680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 914/1871] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:52.481914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a166-2e93-4fae-b625-f29ee7ac5b93 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:52.487033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fde3e7-5693-407b-bedd-19a1ddb5dbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:52.487934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fde3e7-5693-407b-bedd-19a1ddb5dbc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:52.488579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fde3e7-5693-407b-bedd-19a1ddb5dbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:52.489020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fde3e7-5693-407b-bedd-19a1ddb5dbc0 admin admin] PATH_INFO: `/v3/projects/5bccb2a3581d45a2b77d2316c176a614/users/33e42c143f904c83a68a99cb5da98454/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:52.507170 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fde3e7-5693-407b-bedd-19a1ddb5dbc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bccb2a3581d45a2b77d2316c176a614, user_id=33e42c143f904c83a68a99cb5da98454, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:52.510188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fde3e7-5693-407b-bedd-19a1ddb5dbc0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:52.525815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 958/1872] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:52 2022] PUT /identity/v3/projects/5bccb2a3581d45a2b77d2316c176a614/users/33e42c143f904c83a68a99cb5da98454/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:52.538074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55cc9a-b77c-4bdb-adba-a49ec194a4af None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:52.538369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55cc9a-b77c-4bdb-adba-a49ec194a4af None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:52.538651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55cc9a-b77c-4bdb-adba-a49ec194a4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:52.572322 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5d55cc9a-b77c-4bdb-adba-a49ec194a4af None None] MFA Rules not processed for user `33e42c143f904c83a68a99cb5da98454`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:52.573579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5d55cc9a-b77c-4bdb-adba-a49ec194a4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:52.641792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 915/1873] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:52.647051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fde3e7-5693-407b-bedd-19a1ddb5dbc0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:52.720354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fde3e7-5693-407b-bedd-19a1ddb5dbc0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:52.726035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d7667-0b0a-4e23-9b37-6405a029d58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:52.726605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7667-0b0a-4e23-9b37-6405a029d58c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:52.726887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7667-0b0a-4e23-9b37-6405a029d58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:52.727144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7667-0b0a-4e23-9b37-6405a029d58c admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:52.727809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d7667-0b0a-4e23-9b37-6405a029d58c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:52.729716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d7667-0b0a-4e23-9b37-6405a029d58c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:52.765389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 959/1874] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:52 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:52.771768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55cc9a-b77c-4bdb-adba-a49ec194a4af None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:52.784467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55cc9a-b77c-4bdb-adba-a49ec194a4af None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:52.816701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:52.860949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfd092-d293-4876-8c6c-f0fcfd7b783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:52.861978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfd092-d293-4876-8c6c-f0fcfd7b783f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:52.862378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfd092-d293-4876-8c6c-f0fcfd7b783f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:52.864184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfd092-d293-4876-8c6c-f0fcfd7b783f admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:52.865166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfd092-d293-4876-8c6c-f0fcfd7b783f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:52.867164 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfd092-d293-4876-8c6c-f0fcfd7b783f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:52.919114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 916/1875] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:52 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:52.920429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d7667-0b0a-4e23-9b37-6405a029d58c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:52.934943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d7667-0b0a-4e23-9b37-6405a029d58c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:52.942252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2305ca-b9c1-46f2-86cf-abcc83627cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:52.942911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2305ca-b9c1-46f2-86cf-abcc83627cec admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:52.945662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2305ca-b9c1-46f2-86cf-abcc83627cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:52.945960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2305ca-b9c1-46f2-86cf-abcc83627cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:52.946745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2305ca-b9c1-46f2-86cf-abcc83627cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:52.948325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2305ca-b9c1-46f2-86cf-abcc83627cec admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:52.974046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfd092-d293-4876-8c6c-f0fcfd7b783f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:52.976762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 960/1876] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:53.001669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfd092-d293-4876-8c6c-f0fcfd7b783f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:53.007934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d8d9e-06f0-4196-b156-46bc9097d4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:53.008781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d8d9e-06f0-4196-b156-46bc9097d4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:53.009188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d8d9e-06f0-4196-b156-46bc9097d4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:53.009620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d8d9e-06f0-4196-b156-46bc9097d4c8 admin admin] PATH_INFO: `/v3/projects/ca63dd46061642f29bfd0e91ee6020d1/users/22ca15152871400d849426bb1a22ca02/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:53.037482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d8d9e-06f0-4196-b156-46bc9097d4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca63dd46061642f29bfd0e91ee6020d1, user_id=22ca15152871400d849426bb1a22ca02, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:53.041616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d8d9e-06f0-4196-b156-46bc9097d4c8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:53.061182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 917/1877] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:52 2022] PUT /identity/v3/projects/ca63dd46061642f29bfd0e91ee6020d1/users/22ca15152871400d849426bb1a22ca02/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:53.065177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2305ca-b9c1-46f2-86cf-abcc83627cec admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:53.079873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2305ca-b9c1-46f2-86cf-abcc83627cec admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:53.142601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:53.180137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b069394-4916-42c3-ad9d-34f5646f1e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:53.182051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069394-4916-42c3-ad9d-34f5646f1e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:53.182287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069394-4916-42c3-ad9d-34f5646f1e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:53.182562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069394-4916-42c3-ad9d-34f5646f1e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:53.184307 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b069394-4916-42c3-ad9d-34f5646f1e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:53.186924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b069394-4916-42c3-ad9d-34f5646f1e54 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:53.213313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 961/1878] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:53.213502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d8d9e-06f0-4196-b156-46bc9097d4c8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:53.226705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d8d9e-06f0-4196-b156-46bc9097d4c8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:53.248657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:53.249563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:53.250109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:53.250414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] PATH_INFO: `/v3/projects/ca63dd46061642f29bfd0e91ee6020d1/users/22ca15152871400d849426bb1a22ca02/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:53.256614 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca63dd46061642f29bfd0e91ee6020d1, user_id=22ca15152871400d849426bb1a22ca02, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:53.259700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:53.289077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 918/1879] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:53 2022] PUT /identity/v3/projects/ca63dd46061642f29bfd0e91ee6020d1/users/22ca15152871400d849426bb1a22ca02/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:53.300319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84336bc9-1a08-4e2b-bb9d-558fce28e5fb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:53.302214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84336bc9-1a08-4e2b-bb9d-558fce28e5fb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:53.302691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84336bc9-1a08-4e2b-bb9d-558fce28e5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:53.357628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-84336bc9-1a08-4e2b-bb9d-558fce28e5fb None None] MFA Rules not processed for user `22ca15152871400d849426bb1a22ca02`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:53.361805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:53.361932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-84336bc9-1a08-4e2b-bb9d-558fce28e5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:53.365215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:53.454382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 962/1880] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:53.483954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb8036-d84d-457b-9906-1b60698d66fb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:53.487682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:53.487879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84336bc9-1a08-4e2b-bb9d-558fce28e5fb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:53.489569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-84336bc9-1a08-4e2b-bb9d-558fce28e5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:53.490517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:53.490836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:53.491127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:53.492916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:53.502783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84336bc9-1a08-4e2b-bb9d-558fce28e5fb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:53.507070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966f09d-0072-4fe9-a268-f49d608a7683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:53.507694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966f09d-0072-4fe9-a268-f49d608a7683 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:53.508108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966f09d-0072-4fe9-a268-f49d608a7683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:53.508263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966f09d-0072-4fe9-a268-f49d608a7683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:53.517948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966f09d-0072-4fe9-a268-f49d608a7683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:53.520475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966f09d-0072-4fe9-a268-f49d608a7683 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:53.534844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 963/1881] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:53.583261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:53.587235 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:53.618171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 919/1882] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:37:53.677420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966f09d-0072-4fe9-a268-f49d608a7683 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:53.693117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966f09d-0072-4fe9-a268-f49d608a7683 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:53.704181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5844ddd-b9c2-4535-a6dd-c849be74c44f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:53.704808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5844ddd-b9c2-4535-a6dd-c849be74c44f service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:53.705129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5844ddd-b9c2-4535-a6dd-c849be74c44f service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:53.705483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5844ddd-b9c2-4535-a6dd-c849be74c44f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:53.797797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5844ddd-b9c2-4535-a6dd-c849be74c44f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:53.799247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5844ddd-b9c2-4535-a6dd-c849be74c44f service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:53.810946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 964/1883] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:54.979125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:54.980662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:55.023402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:55.117354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1acc19-8c06-4584-ae5a-e2b57fd39b99 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:55.123212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:55.124334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:55.124943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:55.125458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] PATH_INFO: `/v3/users/afb328ad83c84f728cd87e51c77aeb2c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:55.127019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb328ad83c84f728cd87e51c77aeb2c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:55.129671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:55.245456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb328ad83c84f728cd87e51c77aeb2c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:55.263756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb328ad83c84f728cd87e51c77aeb2c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:55.264408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb328ad83c84f728cd87e51c77aeb2c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:55.292998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb328ad83c84f728cd87e51c77aeb2c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:55.294113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb328ad83c84f728cd87e51c77aeb2c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:55.321791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 920/1884] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:54 2022] DELETE /identity/v3/users/afb328ad83c84f728cd87e51c77aeb2c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:55.326979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eead7-17cd-4359-bda0-47f0316ab04e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:55.327468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eead7-17cd-4359-bda0-47f0316ab04e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:55.327575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eead7-17cd-4359-bda0-47f0316ab04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:55.377578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-513eead7-17cd-4359-bda0-47f0316ab04e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:55.379874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-513eead7-17cd-4359-bda0-47f0316ab04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:55.446151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:55.471236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 965/1885] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:55.479572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:55.483536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:55.576482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a9ba-215c-46b7-aed8-6ef7b787d329 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:55.582359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9eae3-ac66-4fe2-828d-1c574e66e0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:55.584573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9eae3-ac66-4fe2-828d-1c574e66e0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:55.584872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9eae3-ac66-4fe2-828d-1c574e66e0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:55.585151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9eae3-ac66-4fe2-828d-1c574e66e0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:55.595809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9eae3-ac66-4fe2-828d-1c574e66e0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:55.598199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9eae3-ac66-4fe2-828d-1c574e66e0e8 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:55.611810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 921/1886] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:56.056547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eead7-17cd-4359-bda0-47f0316ab04e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:56.058012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-513eead7-17cd-4359-bda0-47f0316ab04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:56.072730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eead7-17cd-4359-bda0-47f0316ab04e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:56.079733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:56.080597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:56.080883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:56.081254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] PATH_INFO: `/v3/projects/850b9d5ad3104cc38fae72462130ca9d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:56.082677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] RBAC: Authorizing `identity:delete_project(project_id=850b9d5ad3104cc38fae72462130ca9d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:56.084678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:56.157113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '850b9d5ad3104cc38fae72462130ca9d', 'request_id': 'req-fb54a3df-4566-4e84-95f7-ae60088b970f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.173790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '850b9d5ad3104cc38fae72462130ca9d', 'request_id': 'req-fb54a3df-4566-4e84-95f7-ae60088b970f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.187339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '850b9d5ad3104cc38fae72462130ca9d', 'request_id': 'req-fb54a3df-4566-4e84-95f7-ae60088b970f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.203749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] The token cache is being invalidate because project 850b9d5ad3104cc38fae72462130ca9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:56.204627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.205369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.209340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 966/1887] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:56 2022] DELETE /identity/v3/projects/850b9d5ad3104cc38fae72462130ca9d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:56.212123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9eae3-ac66-4fe2-828d-1c574e66e0e8 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:56.223912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9eae3-ac66-4fe2-828d-1c574e66e0e8 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:56.285867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:56.297492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:56.298458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:56.298964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:56.299293 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] PATH_INFO: `/v3/users/20b4f2e1b6e74712bd6b97194030bff8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:56.301437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b4f2e1b6e74712bd6b97194030bff8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:56.303439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:56.360919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b4f2e1b6e74712bd6b97194030bff8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.369128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b4f2e1b6e74712bd6b97194030bff8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.369786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b4f2e1b6e74712bd6b97194030bff8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.383224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b4f2e1b6e74712bd6b97194030bff8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.383961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad120b25-448d-4e4c-923c-c84eabf0e412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b4f2e1b6e74712bd6b97194030bff8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:56.400824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 922/1888] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:37:56 2022] DELETE /identity/v3/users/20b4f2e1b6e74712bd6b97194030bff8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:56.852740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:56.854735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:56.868425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54a3df-4566-4e84-95f7-ae60088b970f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:56.934541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:56.935195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:56.935483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:56.935761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] PATH_INFO: `/v3/projects/7f40e4ffa6844b81bbaedd5384ba6b57` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:56.936938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f40e4ffa6844b81bbaedd5384ba6b57)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:56.939086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:56.990169 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f40e4ffa6844b81bbaedd5384ba6b57', 'request_id': 'req-a480603c-549e-4d10-b24a-72772858826b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:57.004812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f40e4ffa6844b81bbaedd5384ba6b57', 'request_id': 'req-a480603c-549e-4d10-b24a-72772858826b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:57.010006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f40e4ffa6844b81bbaedd5384ba6b57', 'request_id': 'req-a480603c-549e-4d10-b24a-72772858826b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:57.021707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] The token cache is being invalidate because project 7f40e4ffa6844b81bbaedd5384ba6b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:37:57.021993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:57.022642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a480603c-549e-4d10-b24a-72772858826b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:37:57.024162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 967/1889] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:37:56 2022] DELETE /identity/v3/projects/7f40e4ffa6844b81bbaedd5384ba6b57 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:57.029237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbce5b-af18-4dd5-af99-6c89d02a497b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.029510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbce5b-af18-4dd5-af99-6c89d02a497b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.029794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbce5b-af18-4dd5-af99-6c89d02a497b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.053830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c9bbce5b-af18-4dd5-af99-6c89d02a497b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:57.055054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c9bbce5b-af18-4dd5-af99-6c89d02a497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:57.116163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 923/1890] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:57.124158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd0ea-9243-4795-ac6b-1a948df2b110 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.124461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd0ea-9243-4795-ac6b-1a948df2b110 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.124730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd0ea-9243-4795-ac6b-1a948df2b110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.132671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:57.147914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f50fd0ea-9243-4795-ac6b-1a948df2b110 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:57.149154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f50fd0ea-9243-4795-ac6b-1a948df2b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:57.157143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 968/1891] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:57.165608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbce5b-af18-4dd5-af99-6c89d02a497b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.179402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbce5b-af18-4dd5-af99-6c89d02a497b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.184149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.184773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.185047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.185371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.186069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:37:57.186371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.187957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.200450 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 924/1892] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:37:57.204795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fd0ea-9243-4795-ac6b-1a948df2b110 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.214889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fd0ea-9243-4795-ac6b-1a948df2b110 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.219552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0491dc-3b9e-4349-97e3-1560340540f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.220162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0491dc-3b9e-4349-97e3-1560340540f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.220437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0491dc-3b9e-4349-97e3-1560340540f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.220704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0491dc-3b9e-4349-97e3-1560340540f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.221392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0491dc-3b9e-4349-97e3-1560340540f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.223939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0491dc-3b9e-4349-97e3-1560340540f8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.252100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 969/1893] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:57 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:57.256304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.266610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79eda2c-13f6-4a61-85d9-712bb2d6e521 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.330712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a604d3d-3f78-4f84-8313-2d0e01de0151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.331117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a604d3d-3f78-4f84-8313-2d0e01de0151 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.331336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a604d3d-3f78-4f84-8313-2d0e01de0151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.332166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a604d3d-3f78-4f84-8313-2d0e01de0151 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.332869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a604d3d-3f78-4f84-8313-2d0e01de0151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.334797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a604d3d-3f78-4f84-8313-2d0e01de0151 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.343351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:57.364978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 925/1894] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:57 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:37:57.369225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0491dc-3b9e-4349-97e3-1560340540f8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.380835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0491dc-3b9e-4349-97e3-1560340540f8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.385457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf57822-1c3d-4b78-9d0c-e8bc4c930354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.386120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf57822-1c3d-4b78-9d0c-e8bc4c930354 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.386407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf57822-1c3d-4b78-9d0c-e8bc4c930354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.386682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf57822-1c3d-4b78-9d0c-e8bc4c930354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.389016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf57822-1c3d-4b78-9d0c-e8bc4c930354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.389240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf57822-1c3d-4b78-9d0c-e8bc4c930354 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.401334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 970/1895] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:57.404989 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a604d3d-3f78-4f84-8313-2d0e01de0151 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.416316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a604d3d-3f78-4f84-8313-2d0e01de0151 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.423005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073b635-ed5f-4b3e-a1a0-fc3c4f8815cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.423826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073b635-ed5f-4b3e-a1a0-fc3c4f8815cb admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.423977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073b635-ed5f-4b3e-a1a0-fc3c4f8815cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.424446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073b635-ed5f-4b3e-a1a0-fc3c4f8815cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.426780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073b635-ed5f-4b3e-a1a0-fc3c4f8815cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.428403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073b635-ed5f-4b3e-a1a0-fc3c4f8815cb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.445461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 926/1896] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:57.448242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf57822-1c3d-4b78-9d0c-e8bc4c930354 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.464839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf57822-1c3d-4b78-9d0c-e8bc4c930354 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.470111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303368ab-5db3-4779-b397-766294578e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.470930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303368ab-5db3-4779-b397-766294578e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.471596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303368ab-5db3-4779-b397-766294578e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.471596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303368ab-5db3-4779-b397-766294578e14 admin admin] PATH_INFO: `/v3/projects/3c10e6a732434264ba3bfdb1a3818069/users/655265f556154230b63ecb0bf40de3eb/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.488012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303368ab-5db3-4779-b397-766294578e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c10e6a732434264ba3bfdb1a3818069, user_id=655265f556154230b63ecb0bf40de3eb, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.490739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303368ab-5db3-4779-b397-766294578e14 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.505902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 971/1897] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:57 2022] PUT /identity/v3/projects/3c10e6a732434264ba3bfdb1a3818069/users/655265f556154230b63ecb0bf40de3eb/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:57.515961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d6ede-17d2-400e-9b16-f2a7ed06fb20 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.516253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d6ede-17d2-400e-9b16-f2a7ed06fb20 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.516533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d6ede-17d2-400e-9b16-f2a7ed06fb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.544953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3d7d6ede-17d2-400e-9b16-f2a7ed06fb20 None None] MFA Rules not processed for user `655265f556154230b63ecb0bf40de3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:57.546192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3d7d6ede-17d2-400e-9b16-f2a7ed06fb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:57.610428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 927/1898] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:37:57.614900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303368ab-5db3-4779-b397-766294578e14 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.675676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303368ab-5db3-4779-b397-766294578e14 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.680217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef42a4-66a1-4e8e-b6c4-982167ea2aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.680827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef42a4-66a1-4e8e-b6c4-982167ea2aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.681106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef42a4-66a1-4e8e-b6c4-982167ea2aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.681412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef42a4-66a1-4e8e-b6c4-982167ea2aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.682082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef42a4-66a1-4e8e-b6c4-982167ea2aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.683993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef42a4-66a1-4e8e-b6c4-982167ea2aa6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.710998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 972/1899] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:37:57 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:57.715067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d6ede-17d2-400e-9b16-f2a7ed06fb20 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.727245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d6ede-17d2-400e-9b16-f2a7ed06fb20 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.787938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975e6db-1711-4604-a494-6d47491f8240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.788571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975e6db-1711-4604-a494-6d47491f8240 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.788855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975e6db-1711-4604-a494-6d47491f8240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.789130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975e6db-1711-4604-a494-6d47491f8240 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.789832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975e6db-1711-4604-a494-6d47491f8240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.791741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975e6db-1711-4604-a494-6d47491f8240 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.814161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 928/1900] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:37:57 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:37:57.819056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef42a4-66a1-4e8e-b6c4-982167ea2aa6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.829735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef42a4-66a1-4e8e-b6c4-982167ea2aa6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.836071 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ce208-afa2-4566-a6f9-e0db0de3666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.837091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ce208-afa2-4566-a6f9-e0db0de3666a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.837592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ce208-afa2-4566-a6f9-e0db0de3666a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.837948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ce208-afa2-4566-a6f9-e0db0de3666a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.838836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ce208-afa2-4566-a6f9-e0db0de3666a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.841200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ce208-afa2-4566-a6f9-e0db0de3666a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.852706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 973/1901] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:57.856783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975e6db-1711-4604-a494-6d47491f8240 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.866725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975e6db-1711-4604-a494-6d47491f8240 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.871261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c61a0-a4cd-46cb-b3a5-7911222f9be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.871919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c61a0-a4cd-46cb-b3a5-7911222f9be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.872199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c61a0-a4cd-46cb-b3a5-7911222f9be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.872479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c61a0-a4cd-46cb-b3a5-7911222f9be5 admin admin] PATH_INFO: `/v3/projects/6448f8ef332b4a2cb4211e373dffb9e5/users/ef96ab5e02c644bea3bd5c1606964291/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.888206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c61a0-a4cd-46cb-b3a5-7911222f9be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6448f8ef332b4a2cb4211e373dffb9e5, user_id=ef96ab5e02c644bea3bd5c1606964291, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.890899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c61a0-a4cd-46cb-b3a5-7911222f9be5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.905978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 929/1902] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:57 2022] PUT /identity/v3/projects/6448f8ef332b4a2cb4211e373dffb9e5/users/ef96ab5e02c644bea3bd5c1606964291/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:37:57.910108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ce208-afa2-4566-a6f9-e0db0de3666a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:57.920267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ce208-afa2-4566-a6f9-e0db0de3666a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:57.978759 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce892ec8-a1f4-418d-9248-4186aa54abab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:57.979394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce892ec8-a1f4-418d-9248-4186aa54abab admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:57.979676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce892ec8-a1f4-418d-9248-4186aa54abab admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:57.979948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce892ec8-a1f4-418d-9248-4186aa54abab admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:57.980645 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce892ec8-a1f4-418d-9248-4186aa54abab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:57.982249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce892ec8-a1f4-418d-9248-4186aa54abab admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:57.992473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 974/1903] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:37:57.996914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c61a0-a4cd-46cb-b3a5-7911222f9be5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:58.008136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c61a0-a4cd-46cb-b3a5-7911222f9be5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:58.029130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:58.029919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:58.030220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:58.030816 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] PATH_INFO: `/v3/projects/6448f8ef332b4a2cb4211e373dffb9e5/users/ef96ab5e02c644bea3bd5c1606964291/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:58.034735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6448f8ef332b4a2cb4211e373dffb9e5, user_id=ef96ab5e02c644bea3bd5c1606964291, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:58.039022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:58.051689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 930/1904] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:37:57 2022] PUT /identity/v3/projects/6448f8ef332b4a2cb4211e373dffb9e5/users/ef96ab5e02c644bea3bd5c1606964291/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:37:58.062029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a026e-3a6b-438c-8342-72f3a8d951f8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:58.062759 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a026e-3a6b-438c-8342-72f3a8d951f8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:58.063191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a026e-3a6b-438c-8342-72f3a8d951f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:58.092781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:58.110753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c73a026e-3a6b-438c-8342-72f3a8d951f8 None None] MFA Rules not processed for user `ef96ab5e02c644bea3bd5c1606964291`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:58.112156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c73a026e-3a6b-438c-8342-72f3a8d951f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:58.200628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 975/1905] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:37:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:58.212193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:58.213971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:58.240034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:37:58.283260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a528-0fe2-4c52-8b89-62bfdde2e3e6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:58.288240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f38214-6924-499a-9fa2-547dc8a91c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:58.289400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f38214-6924-499a-9fa2-547dc8a91c94 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:58.289400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f38214-6924-499a-9fa2-547dc8a91c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:58.289790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f38214-6924-499a-9fa2-547dc8a91c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:58.356073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f38214-6924-499a-9fa2-547dc8a91c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:58.358205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f38214-6924-499a-9fa2-547dc8a91c94 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:58.369921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 931/1906] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:37:59.399820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5735fb-78c3-4cd8-a517-5817393f9741 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:59.401254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5735fb-78c3-4cd8-a517-5817393f9741 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:59.401846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5735fb-78c3-4cd8-a517-5817393f9741 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:59.404075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 976/1907] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:37:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:37:59.433589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a398b-4e7f-4c46-b363-7c09692f9eef None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:59.434495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a398b-4e7f-4c46-b363-7c09692f9eef None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:59.435077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a398b-4e7f-4c46-b363-7c09692f9eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:59.464868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7d4a398b-4e7f-4c46-b363-7c09692f9eef None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:37:59.466268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7d4a398b-4e7f-4c46-b363-7c09692f9eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:59.561554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 932/1908] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:37:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:37:59.575119 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5735fb-78c3-4cd8-a517-5817393f9741 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:37:59.577064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-df5735fb-78c3-4cd8-a517-5817393f9741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:37:59.597674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5735fb-78c3-4cd8-a517-5817393f9741 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:37:59.603801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066e228-5356-41db-82fa-0f8826799254 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:37:59.604428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066e228-5356-41db-82fa-0f8826799254 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:37:59.604878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066e228-5356-41db-82fa-0f8826799254 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:37:59.605156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066e228-5356-41db-82fa-0f8826799254 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:37:59.615206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066e228-5356-41db-82fa-0f8826799254 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:37:59.616771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066e228-5356-41db-82fa-0f8826799254 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:37:59.629319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 977/1909] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:37:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:04.438763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a398b-4e7f-4c46-b363-7c09692f9eef None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:04.510623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a398b-4e7f-4c46-b363-7c09692f9eef None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:04.515224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:04.515899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:04.516177 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:04.516451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] PATH_INFO: `/v3/users/655265f556154230b63ecb0bf40de3eb` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:04.517645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=655265f556154230b63ecb0bf40de3eb)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:04.520790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:04.528806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:04.588454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655265f556154230b63ecb0bf40de3eb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:04.598549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655265f556154230b63ecb0bf40de3eb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:04.599289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655265f556154230b63ecb0bf40de3eb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:04.613089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655265f556154230b63ecb0bf40de3eb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:04.614025 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655265f556154230b63ecb0bf40de3eb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:04.626471 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 933/1910] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:04 2022] DELETE /identity/v3/users/655265f556154230b63ecb0bf40de3eb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:04.631722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20b8cf-69e3-4d1e-ad83-c03137d5ffae None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:04.632199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20b8cf-69e3-4d1e-ad83-c03137d5ffae None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:04.632293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20b8cf-69e3-4d1e-ad83-c03137d5ffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:04.654379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3e20b8cf-69e3-4d1e-ad83-c03137d5ffae None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:04.655744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3e20b8cf-69e3-4d1e-ad83-c03137d5ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:04.754233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 978/1911] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:04.775365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:04.776839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:04.871648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4552f-bdf1-42af-9354-392954ccc7ed admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:04.876490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb641d6-4ec0-42e3-8e39-f5e338260722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:04.877680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb641d6-4ec0-42e3-8e39-f5e338260722 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:04.878961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb641d6-4ec0-42e3-8e39-f5e338260722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:04.879577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb641d6-4ec0-42e3-8e39-f5e338260722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:04.892026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb641d6-4ec0-42e3-8e39-f5e338260722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:04.898667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb641d6-4ec0-42e3-8e39-f5e338260722 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:04.914342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 934/1912] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:05.342523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20b8cf-69e3-4d1e-ad83-c03137d5ffae None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:05.344248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3e20b8cf-69e3-4d1e-ad83-c03137d5ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:05.359148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20b8cf-69e3-4d1e-ad83-c03137d5ffae None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:05.367290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:05.368595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:05.369580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:05.370205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] PATH_INFO: `/v3/projects/3c10e6a732434264ba3bfdb1a3818069` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:05.371599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c10e6a732434264ba3bfdb1a3818069)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:05.382101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:05.463977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c10e6a732434264ba3bfdb1a3818069', 'request_id': 'req-97105595-4d45-4e0c-9e38-c8382943d90d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.472910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c10e6a732434264ba3bfdb1a3818069', 'request_id': 'req-97105595-4d45-4e0c-9e38-c8382943d90d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.486841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c10e6a732434264ba3bfdb1a3818069', 'request_id': 'req-97105595-4d45-4e0c-9e38-c8382943d90d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.503394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] The token cache is being invalidate because project 3c10e6a732434264ba3bfdb1a3818069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:05.503890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.504764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.506974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 979/1913] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:05 2022] DELETE /identity/v3/projects/3c10e6a732434264ba3bfdb1a3818069 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:05.510957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb641d6-4ec0-42e3-8e39-f5e338260722 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:05.525712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb641d6-4ec0-42e3-8e39-f5e338260722 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:05.616833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:05.617950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:05.618475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:05.618964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] PATH_INFO: `/v3/users/ef96ab5e02c644bea3bd5c1606964291` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:05.620888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef96ab5e02c644bea3bd5c1606964291)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:05.624575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:05.688844 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:05.722977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef96ab5e02c644bea3bd5c1606964291'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.735855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef96ab5e02c644bea3bd5c1606964291'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.737131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef96ab5e02c644bea3bd5c1606964291'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.770610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef96ab5e02c644bea3bd5c1606964291'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.771443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4b79fefd-84b8-479c-afc8-a4a8917499fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef96ab5e02c644bea3bd5c1606964291'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:05.785722 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 935/1914] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:05 2022] DELETE /identity/v3/users/ef96ab5e02c644bea3bd5c1606964291 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:05.862065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:05.864359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:05.873556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:05.877713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97105595-4d45-4e0c-9e38-c8382943d90d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:05.948600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:05.949496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:05.949763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:05.950378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] PATH_INFO: `/v3/users/1c3e82a7c47d4d4499fab08b2cf30854` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:05.952149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3e82a7c47d4d4499fab08b2cf30854)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:05.954477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:06.027710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3e82a7c47d4d4499fab08b2cf30854'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.045810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3e82a7c47d4d4499fab08b2cf30854'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.082595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3e82a7c47d4d4499fab08b2cf30854'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.083959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3e82a7c47d4d4499fab08b2cf30854'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.098784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3e82a7c47d4d4499fab08b2cf30854'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.106491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 980/1915] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:05 2022] DELETE /identity/v3/users/1c3e82a7c47d4d4499fab08b2cf30854 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:06.111243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86649d10-1073-4c70-9971-0644a107add1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:06.111557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86649d10-1073-4c70-9971-0644a107add1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:06.111842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86649d10-1073-4c70-9971-0644a107add1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:06.139625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-86649d10-1073-4c70-9971-0644a107add1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:06.142405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-86649d10-1073-4c70-9971-0644a107add1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:06.239479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 936/1916] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:06.249965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:06.251761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:06.257693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86649d10-1073-4c70-9971-0644a107add1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:06.259458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-86649d10-1073-4c70-9971-0644a107add1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:06.284018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06732f8c-ebae-4b04-a82d-3f29e754b7d1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:06.321606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:06.323174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:06.323652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:06.324130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] PATH_INFO: `/v3/projects/6448f8ef332b4a2cb4211e373dffb9e5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:06.326295 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6448f8ef332b4a2cb4211e373dffb9e5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:06.329452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:06.398237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86649d10-1073-4c70-9971-0644a107add1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:06.420297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b6e01-0e78-420c-acda-7290129400f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:06.421099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6e01-0e78-420c-acda-7290129400f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:06.422599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6e01-0e78-420c-acda-7290129400f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:06.422960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6e01-0e78-420c-acda-7290129400f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:06.443385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b6e01-0e78-420c-acda-7290129400f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:06.453467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6448f8ef332b4a2cb4211e373dffb9e5', 'request_id': 'req-170dc3ad-27d8-48b8-88c0-180073dbee26'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.453659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b6e01-0e78-420c-acda-7290129400f3 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:06.471240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 937/1917] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:06.492721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6448f8ef332b4a2cb4211e373dffb9e5', 'request_id': 'req-170dc3ad-27d8-48b8-88c0-180073dbee26'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.504768 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6448f8ef332b4a2cb4211e373dffb9e5', 'request_id': 'req-170dc3ad-27d8-48b8-88c0-180073dbee26'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.521284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] The token cache is being invalidate because project 6448f8ef332b4a2cb4211e373dffb9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:06.521685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.528669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-170dc3ad-27d8-48b8-88c0-180073dbee26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:06.538824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 981/1918] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:06 2022] DELETE /identity/v3/projects/6448f8ef332b4a2cb4211e373dffb9e5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:06.545121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084db4f3-d126-47ac-b4ec-5a76be81497f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:06.546056 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084db4f3-d126-47ac-b4ec-5a76be81497f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:06.546500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084db4f3-d126-47ac-b4ec-5a76be81497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:06.583863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-084db4f3-d126-47ac-b4ec-5a76be81497f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:06.585126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-084db4f3-d126-47ac-b4ec-5a76be81497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:06.665404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:06.702633 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 938/1919] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:06.713325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdfd6-dafb-4a35-946f-0d76b3319122 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:06.713839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdfd6-dafb-4a35-946f-0d76b3319122 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:06.714739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdfd6-dafb-4a35-946f-0d76b3319122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:06.734798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:06.741137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-440cdfd6-dafb-4a35-946f-0d76b3319122 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:06.743200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-440cdfd6-dafb-4a35-946f-0d76b3319122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:06.753635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 982/1920] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:06.757935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084db4f3-d126-47ac-b4ec-5a76be81497f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:06.769604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084db4f3-d126-47ac-b4ec-5a76be81497f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:06.791225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:06.791994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:06.792381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:06.792786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:06.793674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:06.794179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:06.796538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:06.811137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 939/1921] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:38:06.815286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cdfd6-dafb-4a35-946f-0d76b3319122 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:06.826696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cdfd6-dafb-4a35-946f-0d76b3319122 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:06.832046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:06.832664 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:06.832945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:06.833212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:06.833991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:06.836615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:06.875396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 983/1922] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:06 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:06.876576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:06.888846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25a4f-4729-4128-b809-73b639f0ba01 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:06.901346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:06.903498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.020606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.021569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.022180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.022600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.023423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.025438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.027818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af126d-d994-48e6-a436-acc78ebf146a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.032804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.033886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.035204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.035493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] PATH_INFO: `/v3/projects/b9fa6550ed7d4062bbd3e8083e7bdc22` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.037709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fa6550ed7d4062bbd3e8083e7bdc22)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.040001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.052411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 940/1923] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:06 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:07.053610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.069854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.080674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ef78a-ea39-44a7-87cd-b01c4bbe1c90 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.092825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fa6550ed7d4062bbd3e8083e7bdc22', 'request_id': 'req-80a4433b-1de0-458c-bd1c-8d7a55465b71'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.100377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fa6550ed7d4062bbd3e8083e7bdc22', 'request_id': 'req-80a4433b-1de0-458c-bd1c-8d7a55465b71'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.109925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fa6550ed7d4062bbd3e8083e7bdc22', 'request_id': 'req-80a4433b-1de0-458c-bd1c-8d7a55465b71'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.122517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] The token cache is being invalidate because project b9fa6550ed7d4062bbd3e8083e7bdc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:07.122858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.123519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.125143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 984/1924] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:06 2022] DELETE /identity/v3/projects/b9fa6550ed7d4062bbd3e8083e7bdc22 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:07.126072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.127471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.150234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.150958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.151245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.151521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] PATH_INFO: `/v3/users/33e42c143f904c83a68a99cb5da98454` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.152717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e42c143f904c83a68a99cb5da98454)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.154677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.226076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e42c143f904c83a68a99cb5da98454'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.232315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4433b-1de0-458c-bd1c-8d7a55465b71 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.238126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.238765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.239099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.239449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.240191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.242031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.250346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e42c143f904c83a68a99cb5da98454'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.251423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e42c143f904c83a68a99cb5da98454'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.257643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 985/1925] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:07.257643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.260857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.272787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:07.278401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:07.279163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45498-d4b2-4b9c-8241-32dae0ee513a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.281840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e42c143f904c83a68a99cb5da98454'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.282566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e42c143f904c83a68a99cb5da98454'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.287922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.288757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.289158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.290046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] PATH_INFO: `/v3/users/667acf048de749ea8988da92b835c240` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.291889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=667acf048de749ea8988da92b835c240)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.294188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.317035 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 941/1926] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:07 2022] DELETE /identity/v3/users/33e42c143f904c83a68a99cb5da98454 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:07.318703 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.321843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.373396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667acf048de749ea8988da92b835c240'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.386225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667acf048de749ea8988da92b835c240'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.399349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667acf048de749ea8988da92b835c240'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.400380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667acf048de749ea8988da92b835c240'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.414694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7dc69586-59a4-46a3-8c56-ccf94c2dda9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667acf048de749ea8988da92b835c240'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:07.422623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 986/1927] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:07 2022] DELETE /identity/v3/users/667acf048de749ea8988da92b835c240 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:07.424118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40280a4f-619f-410b-bcd8-571cd6b92977 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.424428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40280a4f-619f-410b-bcd8-571cd6b92977 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.424736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40280a4f-619f-410b-bcd8-571cd6b92977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.452749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-40280a4f-619f-410b-bcd8-571cd6b92977 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:07.452749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-40280a4f-619f-410b-bcd8-571cd6b92977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.454054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb184f-b83e-4e2d-a588-586c032ed972 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.521733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 987/1928] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:07.528338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.529087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.529765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.529910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.530686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.531350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40280a4f-619f-410b-bcd8-571cd6b92977 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.532482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.532781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-40280a4f-619f-410b-bcd8-571cd6b92977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.544927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 942/1929] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:07.549534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.553546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.572255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152766b-371e-4db1-a7ae-045c57780b77 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.579256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050618f-ea77-4682-81fd-58accfbb0775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.580564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050618f-ea77-4682-81fd-58accfbb0775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.581058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050618f-ea77-4682-81fd-58accfbb0775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.582426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050618f-ea77-4682-81fd-58accfbb0775 admin admin] PATH_INFO: `/v3/projects/e433dc0f008b4e89808cae24d9800a2d/users/cf15541af8d548cdadca05f522291aa8/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.601271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050618f-ea77-4682-81fd-58accfbb0775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e433dc0f008b4e89808cae24d9800a2d, user_id=cf15541af8d548cdadca05f522291aa8, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.603784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40280a4f-619f-410b-bcd8-571cd6b92977 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.603979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050618f-ea77-4682-81fd-58accfbb0775 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.607927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6b1e8-917a-40ca-adc7-447b7c9d5adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.609330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6b1e8-917a-40ca-adc7-447b7c9d5adf service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.609905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6b1e8-917a-40ca-adc7-447b7c9d5adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.610404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6b1e8-917a-40ca-adc7-447b7c9d5adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.621344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 943/1930] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:07 2022] PUT /identity/v3/projects/e433dc0f008b4e89808cae24d9800a2d/users/cf15541af8d548cdadca05f522291aa8/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:07.623045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6b1e8-917a-40ca-adc7-447b7c9d5adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.624579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6b1e8-917a-40ca-adc7-447b7c9d5adf service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.632267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fd21e-b265-4cdf-a453-62422120f687 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.633574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fd21e-b265-4cdf-a453-62422120f687 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.633925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fd21e-b265-4cdf-a453-62422120f687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.637905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 988/1931] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:07.664243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-663fd21e-b265-4cdf-a453-62422120f687 None None] MFA Rules not processed for user `cf15541af8d548cdadca05f522291aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:07.665568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-663fd21e-b265-4cdf-a453-62422120f687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.741239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:07.778919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 944/1932] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:07.780844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6b1e8-917a-40ca-adc7-447b7c9d5adf service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.797607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6b1e8-917a-40ca-adc7-447b7c9d5adf service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.801678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fd21e-b265-4cdf-a453-62422120f687 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.803552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-663fd21e-b265-4cdf-a453-62422120f687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:07.841873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:07.922472 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835db3d0-761a-4892-8d67-aa734a4a9122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.923257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835db3d0-761a-4892-8d67-aa734a4a9122 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.923551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835db3d0-761a-4892-8d67-aa734a4a9122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.923830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835db3d0-761a-4892-8d67-aa734a4a9122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.924511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835db3d0-761a-4892-8d67-aa734a4a9122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.927511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835db3d0-761a-4892-8d67-aa734a4a9122 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:07.966499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fd21e-b265-4cdf-a453-62422120f687 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:07.973829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:07.974516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 989/1933] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:07 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:07.977934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:07.978276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:07.978566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] PATH_INFO: `/v3/projects/acc8de87dc104acbae3e42246993aaf5` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:07.978908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835db3d0-761a-4892-8d67-aa734a4a9122 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:07.980038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc8de87dc104acbae3e42246993aaf5)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:07.982084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.145636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc8de87dc104acbae3e42246993aaf5', 'request_id': 'req-94733dca-e721-4748-8fdf-6731cfcca9aa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:08.153021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835db3d0-761a-4892-8d67-aa734a4a9122 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.162363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc8de87dc104acbae3e42246993aaf5', 'request_id': 'req-94733dca-e721-4748-8fdf-6731cfcca9aa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:08.187058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc8de87dc104acbae3e42246993aaf5', 'request_id': 'req-94733dca-e721-4748-8fdf-6731cfcca9aa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:08.218925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] The token cache is being invalidate because project acc8de87dc104acbae3e42246993aaf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:08.219237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:08.219971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-94733dca-e721-4748-8fdf-6731cfcca9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:08.229851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 945/1934] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:07 2022] DELETE /identity/v3/projects/acc8de87dc104acbae3e42246993aaf5 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:08.231716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c5689-574c-4ebc-bfec-384372a53ad6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.232016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c5689-574c-4ebc-bfec-384372a53ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.232404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c5689-574c-4ebc-bfec-384372a53ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.300567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.302054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.302349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.302856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.303132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a88c5689-574c-4ebc-bfec-384372a53ad6 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:08.303312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.305040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a88c5689-574c-4ebc-bfec-384372a53ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:08.305797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.341787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 946/1935] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:08.343326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7c69b-fcb3-4ab0-9087-d142f87c9c2b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.343659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7c69b-fcb3-4ab0-9087-d142f87c9c2b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.344002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7c69b-fcb3-4ab0-9087-d142f87c9c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.356859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 990/1936] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:07 2022] POST /identity/v3/users => generated 293 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:08.361458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.370635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e1b7c69b-fcb3-4ab0-9087-d142f87c9c2b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:08.372320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e1b7c69b-fcb3-4ab0-9087-d142f87c9c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:08.380155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:08.382586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 947/1937] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:08.387143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7c69b-fcb3-4ab0-9087-d142f87c9c2b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.403762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7c69b-fcb3-4ab0-9087-d142f87c9c2b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.413219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.421467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.421467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.421467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.421467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:08.421467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.421467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.428341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19014efc-00ac-4450-943d-4746197049cf admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.446326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad99ce9-13ae-4770-a04d-03660d11f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.449341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad99ce9-13ae-4770-a04d-03660d11f31f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.449341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad99ce9-13ae-4770-a04d-03660d11f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.449341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad99ce9-13ae-4770-a04d-03660d11f31f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.453838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad99ce9-13ae-4770-a04d-03660d11f31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.462795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad99ce9-13ae-4770-a04d-03660d11f31f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.465323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 948/1938] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:08.470034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.484187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 991/1939] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:08.485378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad99ce9-13ae-4770-a04d-03660d11f31f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.493076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc2c6f-39a9-434e-99b5-a2252a764c4b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.498480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad99ce9-13ae-4770-a04d-03660d11f31f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.506234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d793b-a72b-4a4f-82a6-9e593b9ab1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.507309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d793b-a72b-4a4f-82a6-9e593b9ab1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.507693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d793b-a72b-4a4f-82a6-9e593b9ab1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.508142 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d793b-a72b-4a4f-82a6-9e593b9ab1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.508327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac14e9f-68f5-490f-9745-2ed8490a1533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.508963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac14e9f-68f5-490f-9745-2ed8490a1533 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.509364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d793b-a72b-4a4f-82a6-9e593b9ab1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.509476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac14e9f-68f5-490f-9745-2ed8490a1533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.509564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac14e9f-68f5-490f-9745-2ed8490a1533 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.510256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac14e9f-68f5-490f-9745-2ed8490a1533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.512225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac14e9f-68f5-490f-9745-2ed8490a1533 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.520093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d793b-a72b-4a4f-82a6-9e593b9ab1a5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.536615 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 992/1940] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:08.543293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d793b-a72b-4a4f-82a6-9e593b9ab1a5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.564372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d793b-a72b-4a4f-82a6-9e593b9ab1a5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.572416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473c58c-79d1-4cd1-a28d-7dca705a4700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.573198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473c58c-79d1-4cd1-a28d-7dca705a4700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.573659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473c58c-79d1-4cd1-a28d-7dca705a4700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.574061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473c58c-79d1-4cd1-a28d-7dca705a4700 admin admin] PATH_INFO: `/v3/projects/07fd23a10ca640f4b4cab5f1bedbcbe8/users/9fde6e3126d8419dbca1ee92671ae963/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.584465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 949/1941] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:08 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:08.588547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac14e9f-68f5-490f-9745-2ed8490a1533 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.592350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473c58c-79d1-4cd1-a28d-7dca705a4700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fd23a10ca640f4b4cab5f1bedbcbe8, user_id=9fde6e3126d8419dbca1ee92671ae963, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.595393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473c58c-79d1-4cd1-a28d-7dca705a4700 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.598979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:08.629556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 993/1942] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:08 2022] PUT /identity/v3/projects/07fd23a10ca640f4b4cab5f1bedbcbe8/users/9fde6e3126d8419dbca1ee92671ae963/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:08.632353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe173d-72a0-4468-9b12-180f8dd4c2dc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.635551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe173d-72a0-4468-9b12-180f8dd4c2dc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.636050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe173d-72a0-4468-9b12-180f8dd4c2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.668695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:08.675606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1abe173d-72a0-4468-9b12-180f8dd4c2dc None None] MFA Rules not processed for user `9fde6e3126d8419dbca1ee92671ae963`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:08.677481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1abe173d-72a0-4468-9b12-180f8dd4c2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:08.712515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac14e9f-68f5-490f-9745-2ed8490a1533 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.802707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 994/1943] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:08.803659 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe173d-72a0-4468-9b12-180f8dd4c2dc None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.861315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd9304-ad1a-49a7-bdc0-6b98892383f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.863613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd9304-ad1a-49a7-bdc0-6b98892383f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.863833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd9304-ad1a-49a7-bdc0-6b98892383f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.864741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd9304-ad1a-49a7-bdc0-6b98892383f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.866051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd9304-ad1a-49a7-bdc0-6b98892383f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.870307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd9304-ad1a-49a7-bdc0-6b98892383f0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.918854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 950/1944] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:08 2022] POST /identity/v3/users => generated 282 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:08.936648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd9304-ad1a-49a7-bdc0-6b98892383f0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:08.962359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd9304-ad1a-49a7-bdc0-6b98892383f0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.968532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.969445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.969937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.970533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.972365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.974346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe173d-72a0-4468-9b12-180f8dd4c2dc None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:08.975216 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.986403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5298a9-5653-453d-9213-27f8e4064e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:08.987187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5298a9-5653-453d-9213-27f8e4064e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:08.987687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5298a9-5653-453d-9213-27f8e4064e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:08.988077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5298a9-5653-453d-9213-27f8e4064e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:08.988920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5298a9-5653-453d-9213-27f8e4064e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:08.992870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5298a9-5653-453d-9213-27f8e4064e73 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:08.996409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 951/1945] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:09.002619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.014465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.029146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebc19a-b023-4262-a26e-b1f64f368230 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.036264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.036922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.037206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.037532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] PATH_INFO: `/v3/projects/5bccb2a3581d45a2b77d2316c176a614` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.038755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bccb2a3581d45a2b77d2316c176a614)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.040046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 995/1946] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:08 2022] POST /identity/v3/projects => generated 383 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:09.041050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5298a9-5653-453d-9213-27f8e4064e73 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.041263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.125132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bccb2a3581d45a2b77d2316c176a614', 'request_id': 'req-a7197c4b-9c33-47e7-b0a0-582e334dc089'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.139318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bccb2a3581d45a2b77d2316c176a614', 'request_id': 'req-a7197c4b-9c33-47e7-b0a0-582e334dc089'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.150041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bccb2a3581d45a2b77d2316c176a614', 'request_id': 'req-a7197c4b-9c33-47e7-b0a0-582e334dc089'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.155653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5298a9-5653-453d-9213-27f8e4064e73 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.166707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] The token cache is being invalidate because project 5bccb2a3581d45a2b77d2316c176a614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:09.167024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.167703 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.169248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 952/1947] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:09 2022] DELETE /identity/v3/projects/5bccb2a3581d45a2b77d2316c176a614 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:09.169478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:09.170043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.171507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.228157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.228889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.229249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.229646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.230357 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.231948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.243731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:09.245604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 996/1948] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:09.245604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.250741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.264822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f66d-eacf-4c32-81ce-f6a25d863838 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.265901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7197c4b-9c33-47e7-b0a0-582e334dc089 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.271103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.271935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.272134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.272554 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.272686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.272845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.273124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.273434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] PATH_INFO: `/v3/users/22ca15152871400d849426bb1a22ca02` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.273833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.275269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22ca15152871400d849426bb1a22ca02)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.275756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.277980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.299816 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 953/1949] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:09 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:09.300780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.302734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.315302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694d1e4-b321-4dc2-8c36-5df96084cac6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.321215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b2afc-0fd9-406a-b198-8dea663041fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.322112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b2afc-0fd9-406a-b198-8dea663041fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.322395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b2afc-0fd9-406a-b198-8dea663041fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.322877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b2afc-0fd9-406a-b198-8dea663041fa admin admin] PATH_INFO: `/v3/projects/220747f16fb94fddb4433f830f112eb4/users/b888606719ac4e9dba52d8df2b50fbf0/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.339876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b2afc-0fd9-406a-b198-8dea663041fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=220747f16fb94fddb4433f830f112eb4, user_id=b888606719ac4e9dba52d8df2b50fbf0, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.340760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22ca15152871400d849426bb1a22ca02'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.342568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b2afc-0fd9-406a-b198-8dea663041fa admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.348400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22ca15152871400d849426bb1a22ca02'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.355511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22ca15152871400d849426bb1a22ca02'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.356227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22ca15152871400d849426bb1a22ca02'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.357485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 954/1950] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:09 2022] PUT /identity/v3/projects/220747f16fb94fddb4433f830f112eb4/users/b888606719ac4e9dba52d8df2b50fbf0/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:09.358390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b2afc-0fd9-406a-b198-8dea663041fa admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.368812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c29ca25b-b500-43aa-a98c-1a626a636f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22ca15152871400d849426bb1a22ca02'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:09.374803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 997/1951] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:09 2022] DELETE /identity/v3/users/22ca15152871400d849426bb1a22ca02 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:09.376014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749cd9c-6b60-4c63-8e6e-d5ac75c68de3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.376299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749cd9c-6b60-4c63-8e6e-d5ac75c68de3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.376574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749cd9c-6b60-4c63-8e6e-d5ac75c68de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.406081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4749cd9c-6b60-4c63-8e6e-d5ac75c68de3 None None] MFA Rules not processed for user `b888606719ac4e9dba52d8df2b50fbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:09.407568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4749cd9c-6b60-4c63-8e6e-d5ac75c68de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.450771 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b2afc-0fd9-406a-b198-8dea663041fa admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.512935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 998/1952] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:09 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:09.514111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749cd9c-6b60-4c63-8e6e-d5ac75c68de3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.515602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4749cd9c-6b60-4c63-8e6e-d5ac75c68de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.537459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.538130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.538417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.538726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.539488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.541155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.543049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:09.557806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 955/1953] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:09.559072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.563184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.584183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725442a-80b7-4991-b570-7b9756e0fa0d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.590030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02feb3b-e9b3-4d1f-936f-8a301a7e5390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.590748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02feb3b-e9b3-4d1f-936f-8a301a7e5390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.591102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02feb3b-e9b3-4d1f-936f-8a301a7e5390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.591382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02feb3b-e9b3-4d1f-936f-8a301a7e5390 admin admin] PATH_INFO: `/v3/projects/b2a804d5f2d1411b9942b9790092e0d4/users/3db50c076e4041328c33a824b73fb564/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.611863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02feb3b-e9b3-4d1f-936f-8a301a7e5390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a804d5f2d1411b9942b9790092e0d4, user_id=3db50c076e4041328c33a824b73fb564, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.616001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02feb3b-e9b3-4d1f-936f-8a301a7e5390 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.616945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749cd9c-6b60-4c63-8e6e-d5ac75c68de3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.623591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be1e0-c656-4939-969b-9346ed09c2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.624308 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be1e0-c656-4939-969b-9346ed09c2c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.624655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be1e0-c656-4939-969b-9346ed09c2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.625005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be1e0-c656-4939-969b-9346ed09c2c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.626040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416be1e0-c656-4939-969b-9346ed09c2c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.628869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416be1e0-c656-4939-969b-9346ed09c2c1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.637046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 956/1954] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:09 2022] PUT /identity/v3/projects/b2a804d5f2d1411b9942b9790092e0d4/users/3db50c076e4041328c33a824b73fb564/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:09.641641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02feb3b-e9b3-4d1f-936f-8a301a7e5390 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.689338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 999/1955] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:09 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:09.693525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be1e0-c656-4939-969b-9346ed09c2c1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.694857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:09.738375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02feb3b-e9b3-4d1f-936f-8a301a7e5390 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.800853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be1e0-c656-4939-969b-9346ed09c2c1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.816026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ed861-2818-42cc-85c9-11d409f2f028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.816702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed861-2818-42cc-85c9-11d409f2f028 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.816994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed861-2818-42cc-85c9-11d409f2f028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.817313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed861-2818-42cc-85c9-11d409f2f028 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.817999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ed861-2818-42cc-85c9-11d409f2f028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.820299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ed861-2818-42cc-85c9-11d409f2f028 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.835668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.836469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.836843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.837226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.838102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.839787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.856794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 957/1956] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:38:09.856794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.857944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1000/1957] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:09 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:09.858681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ed861-2818-42cc-85c9-11d409f2f028 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.861236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:09.873265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ed861-2818-42cc-85c9-11d409f2f028 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.883553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704402-1b88-46fb-aa6b-256472941084 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:09.885814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa20cc-c4c2-4f3f-af78-aeb21ae9e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.887261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa20cc-c4c2-4f3f-af78-aeb21ae9e3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.887755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa20cc-c4c2-4f3f-af78-aeb21ae9e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.888289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa20cc-c4c2-4f3f-af78-aeb21ae9e3d6 admin admin] PATH_INFO: `/v3/projects/b2a804d5f2d1411b9942b9790092e0d4/users/3db50c076e4041328c33a824b73fb564/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.892904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:09.892904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:09.892904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:09.892904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] PATH_INFO: `/v3/projects/ca63dd46061642f29bfd0e91ee6020d1` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:09.893432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca63dd46061642f29bfd0e91ee6020d1)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.895153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.906191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa20cc-c4c2-4f3f-af78-aeb21ae9e3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a804d5f2d1411b9942b9790092e0d4, user_id=3db50c076e4041328c33a824b73fb564, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:09.912041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa20cc-c4c2-4f3f-af78-aeb21ae9e3d6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:09.936782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1001/1958] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:09 2022] PUT /identity/v3/projects/b2a804d5f2d1411b9942b9790092e0d4/users/3db50c076e4041328c33a824b73fb564/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:09.937637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa20cc-c4c2-4f3f-af78-aeb21ae9e3d6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:09.980462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca63dd46061642f29bfd0e91ee6020d1', 'request_id': 'req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:10.006237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca63dd46061642f29bfd0e91ee6020d1', 'request_id': 'req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:10.012814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca63dd46061642f29bfd0e91ee6020d1', 'request_id': 'req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:10.017230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa20cc-c4c2-4f3f-af78-aeb21ae9e3d6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.039234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] The token cache is being invalidate because project ca63dd46061642f29bfd0e91ee6020d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:10.039911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:10.040227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:10.044823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e1df2d0e-0a84-4e71-a839-e0b76f2d13c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:10.048821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 958/1959] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:09 2022] DELETE /identity/v3/projects/ca63dd46061642f29bfd0e91ee6020d1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:10.062815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb5cb-46af-4631-bcc1-20991bce67f8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.067490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb5cb-46af-4631-bcc1-20991bce67f8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.068685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb5cb-46af-4631-bcc1-20991bce67f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.099610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39303b81-3e03-4152-865a-d65c9322764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.100412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303b81-3e03-4152-865a-d65c9322764d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.100711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303b81-3e03-4152-865a-d65c9322764d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.100990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303b81-3e03-4152-865a-d65c9322764d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.101734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39303b81-3e03-4152-865a-d65c9322764d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.115388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39303b81-3e03-4152-865a-d65c9322764d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.140635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1002/1960] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:10.140635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3315f4-70af-4fdf-b2f3-c7a42c9ee1f2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.140635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3315f4-70af-4fdf-b2f3-c7a42c9ee1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.140635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3315f4-70af-4fdf-b2f3-c7a42c9ee1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.151023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7e7eb5cb-46af-4631-bcc1-20991bce67f8 None None] MFA Rules not processed for user `3db50c076e4041328c33a824b73fb564`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:10.152377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7e7eb5cb-46af-4631-bcc1-20991bce67f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:10.163917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ea3315f4-70af-4fdf-b2f3-c7a42c9ee1f2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:10.165658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ea3315f4-70af-4fdf-b2f3-c7a42c9ee1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:10.175491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1003/1961] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:10.185545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3315f4-70af-4fdf-b2f3-c7a42c9ee1f2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.186975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ea3315f4-70af-4fdf-b2f3-c7a42c9ee1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:10.200881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3315f4-70af-4fdf-b2f3-c7a42c9ee1f2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.208539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.209601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.210128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.210548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] PATH_INFO: `/v3/projects/e46e1d08568f4176b473f3391b05ae47/users/bdc560781cb745888f5fe8699a1877fb/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.211287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:10.236824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46e1d08568f4176b473f3391b05ae47, user_id=bdc560781cb745888f5fe8699a1877fb, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.240561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.252008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 959/1962] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:10.253504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941275e-a282-4e25-bd2b-351f1a9e3379 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.254316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941275e-a282-4e25-bd2b-351f1a9e3379 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.254745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941275e-a282-4e25-bd2b-351f1a9e3379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.273382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1004/1963] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:10 2022] PUT /identity/v3/projects/e46e1d08568f4176b473f3391b05ae47/users/bdc560781cb745888f5fe8699a1877fb/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:10.274546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.279165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:10.290272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a941275e-a282-4e25-bd2b-351f1a9e3379 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:10.291567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a941275e-a282-4e25-bd2b-351f1a9e3379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:10.376615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 960/1964] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:10.381812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440a694-0082-4de9-a2f4-b12de9689232 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.382150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941275e-a282-4e25-bd2b-351f1a9e3379 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.387262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.387965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.388254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.388534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.390400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:10.395943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941275e-a282-4e25-bd2b-351f1a9e3379 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.401412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237b60c-2622-48bd-8361-937748659763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.402427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237b60c-2622-48bd-8361-937748659763 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.402721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237b60c-2622-48bd-8361-937748659763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.403001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237b60c-2622-48bd-8361-937748659763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.403712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237b60c-2622-48bd-8361-937748659763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.405350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237b60c-2622-48bd-8361-937748659763 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.419012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 961/1965] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:10.419012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237b60c-2622-48bd-8361-937748659763 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.430351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237b60c-2622-48bd-8361-937748659763 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.436668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.437319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.437619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.437901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.441793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:10.442123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.444075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.463630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 962/1966] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:10.464117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.468232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.469773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.477153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471c445-17db-42c4-af53-61e432679a23 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.479502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1005/1967] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:10.480256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.491311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217c850-4053-4c09-b98f-4414e0bef107 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.493934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a6a4c-3e81-4a46-baee-6d3bf6d1d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.495064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a6a4c-3e81-4a46-baee-6d3bf6d1d1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.495567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a6a4c-3e81-4a46-baee-6d3bf6d1d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.496033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a6a4c-3e81-4a46-baee-6d3bf6d1d1f3 admin admin] PATH_INFO: `/v3/projects/e46e1d08568f4176b473f3391b05ae47/users/bdc560781cb745888f5fe8699a1877fb/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.498530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a6a4c-3e81-4a46-baee-6d3bf6d1d1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46e1d08568f4176b473f3391b05ae47, user_id=bdc560781cb745888f5fe8699a1877fb, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.501840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a6a4c-3e81-4a46-baee-6d3bf6d1d1f3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.506144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632fbf7-9b8e-4e8a-a31b-5e5ce0778533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.506955 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632fbf7-9b8e-4e8a-a31b-5e5ce0778533 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.507350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632fbf7-9b8e-4e8a-a31b-5e5ce0778533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.507644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632fbf7-9b8e-4e8a-a31b-5e5ce0778533 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.508317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632fbf7-9b8e-4e8a-a31b-5e5ce0778533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.510294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632fbf7-9b8e-4e8a-a31b-5e5ce0778533 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.515938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 963/1968] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:10 2022] PUT /identity/v3/projects/e46e1d08568f4176b473f3391b05ae47/users/bdc560781cb745888f5fe8699a1877fb/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:10.526443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef02ba-1b58-4b5f-8840-f590132c99d1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.526831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef02ba-1b58-4b5f-8840-f590132c99d1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.527183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef02ba-1b58-4b5f-8840-f590132c99d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.551802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1006/1969] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:10 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:10.552821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632fbf7-9b8e-4e8a-a31b-5e5ce0778533 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.564410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e8ef02ba-1b58-4b5f-8840-f590132c99d1 None None] MFA Rules not processed for user `bdc560781cb745888f5fe8699a1877fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:10.566997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e8ef02ba-1b58-4b5f-8840-f590132c99d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:10.586007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:10.641090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632fbf7-9b8e-4e8a-a31b-5e5ce0778533 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.645401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 964/1970] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:10.652080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3844e2c-fdc2-41a4-aa96-afcc862cac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.652720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844e2c-fdc2-41a4-aa96-afcc862cac18 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.653038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844e2c-fdc2-41a4-aa96-afcc862cac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.653421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844e2c-fdc2-41a4-aa96-afcc862cac18 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.654129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3844e2c-fdc2-41a4-aa96-afcc862cac18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.656368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3844e2c-fdc2-41a4-aa96-afcc862cac18 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.660778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef02ba-1b58-4b5f-8840-f590132c99d1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.662191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e8ef02ba-1b58-4b5f-8840-f590132c99d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:10.685786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1007/1971] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:10 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:10.691314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3844e2c-fdc2-41a4-aa96-afcc862cac18 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.703726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3844e2c-fdc2-41a4-aa96-afcc862cac18 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.708120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:10.709255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7378d3-df9a-4d51-83a8-56aa429bcdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.710181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7378d3-df9a-4d51-83a8-56aa429bcdb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.710453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7378d3-df9a-4d51-83a8-56aa429bcdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.710776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7378d3-df9a-4d51-83a8-56aa429bcdb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.712092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7378d3-df9a-4d51-83a8-56aa429bcdb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.713713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7378d3-df9a-4d51-83a8-56aa429bcdb8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.728336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1008/1972] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:10.729415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7378d3-df9a-4d51-83a8-56aa429bcdb8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.741053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7378d3-df9a-4d51-83a8-56aa429bcdb8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.743566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef02ba-1b58-4b5f-8840-f590132c99d1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.746376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b92c3c-c29b-4f4c-9253-a2094a4e23fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.748092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee574f3-2b41-4033-aec4-743da369fb47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.748938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee574f3-2b41-4033-aec4-743da369fb47 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.749183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee574f3-2b41-4033-aec4-743da369fb47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.749713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee574f3-2b41-4033-aec4-743da369fb47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.749841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b92c3c-c29b-4f4c-9253-a2094a4e23fb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.750549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b92c3c-c29b-4f4c-9253-a2094a4e23fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.753412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b92c3c-c29b-4f4c-9253-a2094a4e23fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.754299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b92c3c-c29b-4f4c-9253-a2094a4e23fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.761396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee574f3-2b41-4033-aec4-743da369fb47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.762825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee574f3-2b41-4033-aec4-743da369fb47 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.765990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b92c3c-c29b-4f4c-9253-a2094a4e23fb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.775927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 965/1973] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:10 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:10.783895 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1009/1974] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:10.788567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee574f3-2b41-4033-aec4-743da369fb47 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.811398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee574f3-2b41-4033-aec4-743da369fb47 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:10.817751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b5491-95fa-4a36-853c-ba962dd7e43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:10.818621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b5491-95fa-4a36-853c-ba962dd7e43b admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.818975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b5491-95fa-4a36-853c-ba962dd7e43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.819601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b5491-95fa-4a36-853c-ba962dd7e43b admin admin] PATH_INFO: `/v3/projects/b83b7980e2224a3e99d1d6ea37f5bbb6/users/cd039b9a1b7143a2b204e694b3cc7b0d/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.838644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b5491-95fa-4a36-853c-ba962dd7e43b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83b7980e2224a3e99d1d6ea37f5bbb6, user_id=cd039b9a1b7143a2b204e694b3cc7b0d, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:10.856588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b5491-95fa-4a36-853c-ba962dd7e43b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:10.881331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 966/1975] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:10 2022] PUT /identity/v3/projects/b83b7980e2224a3e99d1d6ea37f5bbb6/users/cd039b9a1b7143a2b204e694b3cc7b0d/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:10.894447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602491-b4f1-4b20-84a1-6fc7ab6fa0ac None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:10.894741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602491-b4f1-4b20-84a1-6fc7ab6fa0ac None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:10.895023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602491-b4f1-4b20-84a1-6fc7ab6fa0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:10.927656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b5491-95fa-4a36-853c-ba962dd7e43b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:10.947969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b4602491-b4f1-4b20-84a1-6fc7ab6fa0ac None None] MFA Rules not processed for user `cd039b9a1b7143a2b204e694b3cc7b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:10.949359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b4602491-b4f1-4b20-84a1-6fc7ab6fa0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:11.001609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b5491-95fa-4a36-853c-ba962dd7e43b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:11.007367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:11.008029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.008308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.008554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:11.010468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:11.027998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1010/1976] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:11.032340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602491-b4f1-4b20-84a1-6fc7ab6fa0ac None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:11.095804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:11.099109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:11.115651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 967/1977] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:10 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:11.132585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602491-b4f1-4b20-84a1-6fc7ab6fa0ac None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:11.168915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6488d9c-2610-4e83-8fbf-bec3782d5e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:11.169904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6488d9c-2610-4e83-8fbf-bec3782d5e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.170400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6488d9c-2610-4e83-8fbf-bec3782d5e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.170692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6488d9c-2610-4e83-8fbf-bec3782d5e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:11.171389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6488d9c-2610-4e83-8fbf-bec3782d5e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:11.174747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6488d9c-2610-4e83-8fbf-bec3782d5e55 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:11.228534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1011/1978] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:11 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:11.233144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:11.254862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cce49-7c5c-4293-bd33-a2a8b33c7f99 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:11.385366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19af48-d0d5-4697-8004-b8d0a68edebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:11.386026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19af48-d0d5-4697-8004-b8d0a68edebd admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.386312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19af48-d0d5-4697-8004-b8d0a68edebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.386587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19af48-d0d5-4697-8004-b8d0a68edebd admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:11.387465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19af48-d0d5-4697-8004-b8d0a68edebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:11.389581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19af48-d0d5-4697-8004-b8d0a68edebd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:11.407316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:11.434723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 968/1979] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:11 2022] POST /identity/v3/users => generated 295 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:11.438648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6488d9c-2610-4e83-8fbf-bec3782d5e55 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:11.452058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:11.461796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6488d9c-2610-4e83-8fbf-bec3782d5e55 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:11.469149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbf468-0a0b-436b-8650-335364e19ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:11.470030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbf468-0a0b-436b-8650-335364e19ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.470714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbf468-0a0b-436b-8650-335364e19ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.472621 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbf468-0a0b-436b-8650-335364e19ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:11.474539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbf468-0a0b-436b-8650-335364e19ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:11.476291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbf468-0a0b-436b-8650-335364e19ad9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:11.495689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1012/1980] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:11.499753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19af48-d0d5-4697-8004-b8d0a68edebd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:11.511277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19af48-d0d5-4697-8004-b8d0a68edebd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:11.516309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb8d2a-bfa3-4389-bce0-fe25c30924ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:11.516984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb8d2a-bfa3-4389-bce0-fe25c30924ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.517266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb8d2a-bfa3-4389-bce0-fe25c30924ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.517583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb8d2a-bfa3-4389-bce0-fe25c30924ec admin admin] PATH_INFO: `/v3/projects/01e9d5e4d9344c40ac5b44c6adf81943/users/860eea71fb514c1d80f4192067d66512/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:11.539239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb8d2a-bfa3-4389-bce0-fe25c30924ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e9d5e4d9344c40ac5b44c6adf81943, user_id=860eea71fb514c1d80f4192067d66512, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:11.544342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb8d2a-bfa3-4389-bce0-fe25c30924ec admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:11.567586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 969/1981] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:11 2022] PUT /identity/v3/projects/01e9d5e4d9344c40ac5b44c6adf81943/users/860eea71fb514c1d80f4192067d66512/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:11.571540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbf468-0a0b-436b-8650-335364e19ad9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:11.583681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbf468-0a0b-436b-8650-335364e19ad9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:11.643907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d466077-b46f-4ba1-ab7c-f5ad9cc45018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:11.644557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466077-b46f-4ba1-ab7c-f5ad9cc45018 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.644839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466077-b46f-4ba1-ab7c-f5ad9cc45018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.645110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466077-b46f-4ba1-ab7c-f5ad9cc45018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:11.645846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466077-b46f-4ba1-ab7c-f5ad9cc45018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:11.647434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466077-b46f-4ba1-ab7c-f5ad9cc45018 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:11.662854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb8d2a-bfa3-4389-bce0-fe25c30924ec admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:11.663336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1013/1982] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:11.673384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb8d2a-bfa3-4389-bce0-fe25c30924ec admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:11.679001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b8ae-54c9-4af5-be37-465ba4734010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:11.679664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b8ae-54c9-4af5-be37-465ba4734010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.679942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b8ae-54c9-4af5-be37-465ba4734010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.680213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b8ae-54c9-4af5-be37-465ba4734010 admin admin] PATH_INFO: `/v3/projects/01e9d5e4d9344c40ac5b44c6adf81943/users/860eea71fb514c1d80f4192067d66512/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:11.682168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935b8ae-54c9-4af5-be37-465ba4734010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e9d5e4d9344c40ac5b44c6adf81943, user_id=860eea71fb514c1d80f4192067d66512, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:11.685213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935b8ae-54c9-4af5-be37-465ba4734010 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:11.696873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 970/1983] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:11 2022] PUT /identity/v3/projects/01e9d5e4d9344c40ac5b44c6adf81943/users/860eea71fb514c1d80f4192067d66512/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:11.707583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f485-78e9-4d4f-92b5-e6ce233b9173 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.707871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f485-78e9-4d4f-92b5-e6ce233b9173 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.708147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f485-78e9-4d4f-92b5-e6ce233b9173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:11.740781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e9a2f485-78e9-4d4f-92b5-e6ce233b9173 None None] MFA Rules not processed for user `860eea71fb514c1d80f4192067d66512`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:11.742109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e9a2f485-78e9-4d4f-92b5-e6ce233b9173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:11.818318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1014/1984] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:11.829246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b8ae-54c9-4af5-be37-465ba4734010 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:11.929880 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b8ae-54c9-4af5-be37-465ba4734010 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:11.939634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e51bc-4d48-44f6-acb5-b01deefa491b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:11.940836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e51bc-4d48-44f6-acb5-b01deefa491b service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:11.941664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e51bc-4d48-44f6-acb5-b01deefa491b service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:11.941664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e51bc-4d48-44f6-acb5-b01deefa491b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:12.038195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e51bc-4d48-44f6-acb5-b01deefa491b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:12.040128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e51bc-4d48-44f6-acb5-b01deefa491b service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:12.051203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 971/1985] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:12.228834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2f485-78e9-4d4f-92b5-e6ce233b9173 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:12.244548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2f485-78e9-4d4f-92b5-e6ce233b9173 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:12.249875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:12.250559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:12.250831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:12.251101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:12.253062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:12.303616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca94d5-1244-4666-8dde-5575b0de5c9c None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:12.303900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca94d5-1244-4666-8dde-5575b0de5c9c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:12.304140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca94d5-1244-4666-8dde-5575b0de5c9c None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:12.305535 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 972/1986] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:38:12.332454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:12.334080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:12.338160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f742e-da13-4f7f-a4a2-432f1eaccd9a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:12.338784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f742e-da13-4f7f-a4a2-432f1eaccd9a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:12.338784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f742e-da13-4f7f-a4a2-432f1eaccd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:12.350482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1015/1987] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:12.376400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9a2f742e-da13-4f7f-a4a2-432f1eaccd9a None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:12.377092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9a2f742e-da13-4f7f-a4a2-432f1eaccd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:12.529843 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 973/1988] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:38:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:12.545224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:12.546693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:12.579200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7aa7fa-3105-4dce-8525-3c71bbc77692 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:12.596977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c658d3a-f2c1-4a53-9ac5-768f28cc301e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:12.597995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c658d3a-f2c1-4a53-9ac5-768f28cc301e service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:12.598294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c658d3a-f2c1-4a53-9ac5-768f28cc301e service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:12.600257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c658d3a-f2c1-4a53-9ac5-768f28cc301e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:12.624472 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c658d3a-f2c1-4a53-9ac5-768f28cc301e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:12.626575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c658d3a-f2c1-4a53-9ac5-768f28cc301e service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:12.640715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:12.650183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1016/1989] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:13.335057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f742e-da13-4f7f-a4a2-432f1eaccd9a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:13.338186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9a2f742e-da13-4f7f-a4a2-432f1eaccd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:13.354871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:13.473337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f742e-da13-4f7f-a4a2-432f1eaccd9a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:13.478285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:13.478963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:13.479244 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:13.479524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] PATH_INFO: `/v3/users/6f85cf8b029b4b579570e0d8d5c27926` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:13.480702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f85cf8b029b4b579570e0d8d5c27926)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:13.482680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:13.538458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f85cf8b029b4b579570e0d8d5c27926'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.546564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f85cf8b029b4b579570e0d8d5c27926'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.547347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f85cf8b029b4b579570e0d8d5c27926'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.562730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f85cf8b029b4b579570e0d8d5c27926'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.563528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f85cf8b029b4b579570e0d8d5c27926'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.580516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 974/1990] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:13 2022] DELETE /identity/v3/users/6f85cf8b029b4b579570e0d8d5c27926 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:13.693100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c658d3a-f2c1-4a53-9ac5-768f28cc301e service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:13.704693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c658d3a-f2c1-4a53-9ac5-768f28cc301e service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:13.762690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:13.763328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:13.763687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:13.764052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:13.766120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:13.777804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:13.779630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:13.865679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:13.867318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:13.884593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab577a-0899-49b1-9004-78212ebaf849 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:13.884988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1017/1991] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:13.889514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:13.890168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:13.890447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:13.890788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] PATH_INFO: `/v3/projects/195dd8554c7743288381d115015351b5` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:13.891864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=195dd8554c7743288381d115015351b5)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:13.895458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:13.934931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:13.965889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195dd8554c7743288381d115015351b5', 'request_id': 'req-a23137b9-e2f5-40c2-bfa3-33df07e98faf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.983725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195dd8554c7743288381d115015351b5', 'request_id': 'req-a23137b9-e2f5-40c2-bfa3-33df07e98faf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.984831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195dd8554c7743288381d115015351b5', 'request_id': 'req-a23137b9-e2f5-40c2-bfa3-33df07e98faf'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.998057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] The token cache is being invalidate because project 195dd8554c7743288381d115015351b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:13.998290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:13.998974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.000831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 975/1992] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:13 2022] DELETE /identity/v3/projects/195dd8554c7743288381d115015351b5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:14.008566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:14.024324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe94a10-1e83-4b98-86e9-d3361705e6b9 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:14.086860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:14.087534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.087838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.088110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] PATH_INFO: `/v3/users/16e1c92765854bb798963302db0d768d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.089276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e1c92765854bb798963302db0d768d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:14.091337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:14.140456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e1c92765854bb798963302db0d768d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.145168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:14.148355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e1c92765854bb798963302db0d768d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.156727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e1c92765854bb798963302db0d768d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.157431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e1c92765854bb798963302db0d768d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.174211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e1c92765854bb798963302db0d768d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.183606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1018/1993] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:14 2022] DELETE /identity/v3/users/16e1c92765854bb798963302db0d768d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:14.474040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:14.475494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:14.477088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:14.479176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:14.512465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23137b9-e2f5-40c2-bfa3-33df07e98faf admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:14.561945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa484b-e0b1-4ef4-a7eb-e9d24adbc568 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:14.568133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:14.568995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.569567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.569897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] PATH_INFO: `/v3/users/b888606719ac4e9dba52d8df2b50fbf0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.571174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b888606719ac4e9dba52d8df2b50fbf0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:14.573601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:14.584084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:14.584749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.585038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.585321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] PATH_INFO: `/v3/projects/57a8c87f22814acb8bdff213e3267da0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.586503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a8c87f22814acb8bdff213e3267da0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:14.588645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:14.630277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b888606719ac4e9dba52d8df2b50fbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.637872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b888606719ac4e9dba52d8df2b50fbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.642426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a8c87f22814acb8bdff213e3267da0', 'request_id': 'req-b8caf5c7-c3b1-4f57-b514-b91397eadafa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.647960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b888606719ac4e9dba52d8df2b50fbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.648867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b888606719ac4e9dba52d8df2b50fbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.651196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a8c87f22814acb8bdff213e3267da0', 'request_id': 'req-b8caf5c7-c3b1-4f57-b514-b91397eadafa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.657119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a8c87f22814acb8bdff213e3267da0', 'request_id': 'req-b8caf5c7-c3b1-4f57-b514-b91397eadafa'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.666888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5095e4e0-2bec-4017-a018-10d9e20a900e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b888606719ac4e9dba52d8df2b50fbf0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.671723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] The token cache is being invalidate because project 57a8c87f22814acb8bdff213e3267da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:14.672079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.673134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b8caf5c7-c3b1-4f57-b514-b91397eadafa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:14.673581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1019/1994] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:14 2022] DELETE /identity/v3/users/b888606719ac4e9dba52d8df2b50fbf0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:14.674432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 976/1995] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:14 2022] DELETE /identity/v3/projects/57a8c87f22814acb8bdff213e3267da0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:14.678629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411415dc-1779-4b0d-8098-498cfb57d5f9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.678912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411415dc-1779-4b0d-8098-498cfb57d5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.679286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411415dc-1779-4b0d-8098-498cfb57d5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.681466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d241f-9c79-4c8b-83f2-089fe2f1809a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.681895 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d241f-9c79-4c8b-83f2-089fe2f1809a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.682184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d241f-9c79-4c8b-83f2-089fe2f1809a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.699907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-411415dc-1779-4b0d-8098-498cfb57d5f9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:14.701131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-411415dc-1779-4b0d-8098-498cfb57d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:14.705139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ba3d241f-9c79-4c8b-83f2-089fe2f1809a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:14.707776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ba3d241f-9c79-4c8b-83f2-089fe2f1809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:14.752146 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:14.772303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:14.777961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1020/1996] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:14.788998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411415dc-1779-4b0d-8098-498cfb57d5f9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:14.791366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-411415dc-1779-4b0d-8098-498cfb57d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:14.802630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 977/1997] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:14.810318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171757c-4533-40f6-95ba-a7e42313c2b2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.810613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171757c-4533-40f6-95ba-a7e42313c2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.810936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171757c-4533-40f6-95ba-a7e42313c2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.834079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9171757c-4533-40f6-95ba-a7e42313c2b2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:14.835217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9171757c-4533-40f6-95ba-a7e42313c2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:14.844300 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 978/1998] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:14.848518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171757c-4533-40f6-95ba-a7e42313c2b2 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:14.858215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171757c-4533-40f6-95ba-a7e42313c2b2 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:14.863310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:14.863959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.864212 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.864483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.865186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:14.865521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:14.866174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411415dc-1779-4b0d-8098-498cfb57d5f9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:14.867124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:14.870160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb217c-72ed-46a8-a4b8-754fc9d36b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:14.870788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb217c-72ed-46a8-a4b8-754fc9d36b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.871330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb217c-72ed-46a8-a4b8-754fc9d36b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.871609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb217c-72ed-46a8-a4b8-754fc9d36b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.880578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 979/1999] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:38:14.883790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb217c-72ed-46a8-a4b8-754fc9d36b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:14.884868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:14.885256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb217c-72ed-46a8-a4b8-754fc9d36b1a service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:14.896376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801d121-16a5-4673-9a02-54f2126b37bd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:14.898227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1021/2000] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:14.902235 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105ca57-ad6e-4cf1-a637-9a83c28be996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:14.902937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105ca57-ad6e-4cf1-a637-9a83c28be996 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:14.903221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105ca57-ad6e-4cf1-a637-9a83c28be996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:14.903492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105ca57-ad6e-4cf1-a637-9a83c28be996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:14.904158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105ca57-ad6e-4cf1-a637-9a83c28be996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:14.906089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105ca57-ad6e-4cf1-a637-9a83c28be996 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:14.944359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 980/2001] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:14 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:14.946018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb217c-72ed-46a8-a4b8-754fc9d36b1a service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:14.959682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb217c-72ed-46a8-a4b8-754fc9d36b1a service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:15.035675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5669da0-b1a7-41fb-b1e7-9c81c51c47c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:15.036318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5669da0-b1a7-41fb-b1e7-9c81c51c47c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:15.036628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5669da0-b1a7-41fb-b1e7-9c81c51c47c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:15.036937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5669da0-b1a7-41fb-b1e7-9c81c51c47c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:15.037635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5669da0-b1a7-41fb-b1e7-9c81c51c47c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:15.039587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5669da0-b1a7-41fb-b1e7-9c81c51c47c7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:15.073258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1022/2002] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:14 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:15.077331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105ca57-ad6e-4cf1-a637-9a83c28be996 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:15.093488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105ca57-ad6e-4cf1-a637-9a83c28be996 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:15.114478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142d0de-54e8-4209-89fb-a3b9a093146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:15.115124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142d0de-54e8-4209-89fb-a3b9a093146e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:15.115407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142d0de-54e8-4209-89fb-a3b9a093146e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:15.115686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142d0de-54e8-4209-89fb-a3b9a093146e admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:15.116385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142d0de-54e8-4209-89fb-a3b9a093146e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:15.118022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142d0de-54e8-4209-89fb-a3b9a093146e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:15.129581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 981/2003] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:38:15.133539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5669da0-b1a7-41fb-b1e7-9c81c51c47c7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:15.144613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5669da0-b1a7-41fb-b1e7-9c81c51c47c7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:15.149501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:15.150132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:15.150419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:15.150696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:15.151397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:15.152997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:15.164340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1023/2004] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:15.168447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142d0de-54e8-4209-89fb-a3b9a093146e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:15.181960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142d0de-54e8-4209-89fb-a3b9a093146e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:15.189671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5230d-e814-4b9d-a096-874be6e97215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:15.191355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5230d-e814-4b9d-a096-874be6e97215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:15.191689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5230d-e814-4b9d-a096-874be6e97215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:15.192097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5230d-e814-4b9d-a096-874be6e97215 admin admin] PATH_INFO: `/v3/projects/77d1810d277344d6b6aeeb65d134e618/users/dbd454d2a4b542689c0437eb4dbefc02/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:15.208765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5230d-e814-4b9d-a096-874be6e97215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d1810d277344d6b6aeeb65d134e618, user_id=dbd454d2a4b542689c0437eb4dbefc02, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:15.214532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5230d-e814-4b9d-a096-874be6e97215 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:15.231649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 982/2005] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:15 2022] PUT /identity/v3/projects/77d1810d277344d6b6aeeb65d134e618/users/dbd454d2a4b542689c0437eb4dbefc02/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:15.243469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e53297-13a8-4737-956e-7faa125ff794 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:15.243766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e53297-13a8-4737-956e-7faa125ff794 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:15.244001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:15.245189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e53297-13a8-4737-956e-7faa125ff794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:15.246565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:15.267136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfeeb6-1277-4595-acb6-c887e013ab62 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:15.289983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c8e53297-13a8-4737-956e-7faa125ff794 None None] MFA Rules not processed for user `dbd454d2a4b542689c0437eb4dbefc02`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:15.291280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c8e53297-13a8-4737-956e-7faa125ff794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:15.336068 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:15.336854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:15.337138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:15.338710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] PATH_INFO: `/v3/projects/220747f16fb94fddb4433f830f112eb4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:15.338710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=220747f16fb94fddb4433f830f112eb4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:15.341236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:15.354074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:15.361584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 983/2006] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:15.374730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e53297-13a8-4737-956e-7faa125ff794 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:15.377035 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c8e53297-13a8-4737-956e-7faa125ff794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:15.420593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220747f16fb94fddb4433f830f112eb4', 'request_id': 'req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.428506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220747f16fb94fddb4433f830f112eb4', 'request_id': 'req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.446932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220747f16fb94fddb4433f830f112eb4', 'request_id': 'req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.467088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] The token cache is being invalidate because project 220747f16fb94fddb4433f830f112eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:15.467376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.468234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.474779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1024/2007] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:15 2022] DELETE /identity/v3/projects/220747f16fb94fddb4433f830f112eb4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:15.476093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:15.477638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:15.516603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:15.533351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e53297-13a8-4737-956e-7faa125ff794 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:15.573842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3e4d8-d0c1-4b7d-ab83-879187553d1f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:15.579414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:15.580447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:15.581722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:15.582183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] PATH_INFO: `/v3/users/bdc560781cb745888f5fe8699a1877fb` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:15.583980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc560781cb745888f5fe8699a1877fb)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:15.587085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:15.600730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93fd5e-bca5-4451-8a27-06f68b55de54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '649334bb7ed04bd69eb971fc070d96e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:15.601136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fd5e-bca5-4451-8a27-06f68b55de54 service glance] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:15.601457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fd5e-bca5-4451-8a27-06f68b55de54 service glance] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:15.601741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fd5e-bca5-4451-8a27-06f68b55de54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:15.604262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3b93fd5e-bca5-4451-8a27-06f68b55de54 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:15.649344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc560781cb745888f5fe8699a1877fb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.657658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc560781cb745888f5fe8699a1877fb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.666319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc560781cb745888f5fe8699a1877fb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.667304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc560781cb745888f5fe8699a1877fb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.682907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc560781cb745888f5fe8699a1877fb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:15.683918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93fd5e-bca5-4451-8a27-06f68b55de54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:15.685549 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93fd5e-bca5-4451-8a27-06f68b55de54 service glance] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:15.690083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1025/2008] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:15 2022] DELETE /identity/v3/users/bdc560781cb745888f5fe8699a1877fb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:15.697408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 984/2009] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:16.052695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:16.054401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:16.089859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:16.173512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94fd2-3b5c-44b8-a710-66fb0bcbcab0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:16.182924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:16.183972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.184849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.185016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] PATH_INFO: `/v3/projects/e46e1d08568f4176b473f3391b05ae47` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.187016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46e1d08568f4176b473f3391b05ae47)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:16.190918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:16.263967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46e1d08568f4176b473f3391b05ae47', 'request_id': 'req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:16.272353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46e1d08568f4176b473f3391b05ae47', 'request_id': 'req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:16.284798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46e1d08568f4176b473f3391b05ae47', 'request_id': 'req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:16.301905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] The token cache is being invalidate because project e46e1d08568f4176b473f3391b05ae47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:16.305700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:16.306706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-13f9832b-6dd3-4d4b-ac19-6e5f162908b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:16.309261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1026/2010] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:16 2022] DELETE /identity/v3/projects/e46e1d08568f4176b473f3391b05ae47 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:16.314278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546944cf-4e65-4101-be22-b5c3aa6bcb1f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.314573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546944cf-4e65-4101-be22-b5c3aa6bcb1f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.314854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546944cf-4e65-4101-be22-b5c3aa6bcb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.354538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-546944cf-4e65-4101-be22-b5c3aa6bcb1f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:16.368534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-546944cf-4e65-4101-be22-b5c3aa6bcb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:16.430035 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 985/2011] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:16.438546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618f53-5407-4ba2-8aca-b99f9dcd3fb4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.438943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618f53-5407-4ba2-8aca-b99f9dcd3fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.439291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37618f53-5407-4ba2-8aca-b99f9dcd3fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.463093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-37618f53-5407-4ba2-8aca-b99f9dcd3fb4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:16.464950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-37618f53-5407-4ba2-8aca-b99f9dcd3fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:16.477129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1027/2012] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:16.483427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546944cf-4e65-4101-be22-b5c3aa6bcb1f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:16.495997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546944cf-4e65-4101-be22-b5c3aa6bcb1f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:16.501968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:16.504103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.504379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.504654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.505386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:16.505693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:16.507294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:16.520147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 986/2013] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:16.524580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37618f53-5407-4ba2-8aca-b99f9dcd3fb4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:16.534702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37618f53-5407-4ba2-8aca-b99f9dcd3fb4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:16.539585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3be84e-9e63-4351-8628-2f1932316d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:16.540207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be84e-9e63-4351-8628-2f1932316d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.540492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be84e-9e63-4351-8628-2f1932316d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.540988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be84e-9e63-4351-8628-2f1932316d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.541469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3be84e-9e63-4351-8628-2f1932316d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:16.543355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3be84e-9e63-4351-8628-2f1932316d18 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:16.576679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1028/2014] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:16 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:16.577862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:16.588329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c6fd-7861-4199-bc7c-87656bcad175 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:16.624338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:16.657334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55285b-2e44-4830-a5ee-2ec406bf2931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:16.658076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55285b-2e44-4830-a5ee-2ec406bf2931 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.658429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55285b-2e44-4830-a5ee-2ec406bf2931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.658774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55285b-2e44-4830-a5ee-2ec406bf2931 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.660149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55285b-2e44-4830-a5ee-2ec406bf2931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:16.662364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55285b-2e44-4830-a5ee-2ec406bf2931 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:16.686371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 987/2015] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:16 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:16.690606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3be84e-9e63-4351-8628-2f1932316d18 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:16.705408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3be84e-9e63-4351-8628-2f1932316d18 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:16.712041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c99b21-6fee-4df0-9020-5a6f2a261a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:16.712690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c99b21-6fee-4df0-9020-5a6f2a261a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.712950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c99b21-6fee-4df0-9020-5a6f2a261a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.713223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c99b21-6fee-4df0-9020-5a6f2a261a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.713986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c99b21-6fee-4df0-9020-5a6f2a261a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:16.715588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c99b21-6fee-4df0-9020-5a6f2a261a76 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:16.728929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1029/2016] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:38:16.732958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55285b-2e44-4830-a5ee-2ec406bf2931 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:16.744276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55285b-2e44-4830-a5ee-2ec406bf2931 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:16.752742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc70f7c-f5e8-4a6c-a173-35637ccb0b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:16.753454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc70f7c-f5e8-4a6c-a173-35637ccb0b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.753745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc70f7c-f5e8-4a6c-a173-35637ccb0b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.754026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc70f7c-f5e8-4a6c-a173-35637ccb0b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.754736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc70f7c-f5e8-4a6c-a173-35637ccb0b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:16.756385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc70f7c-f5e8-4a6c-a173-35637ccb0b82 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:16.771763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 988/2017] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:16.773062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c99b21-6fee-4df0-9020-5a6f2a261a76 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:16.785707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c99b21-6fee-4df0-9020-5a6f2a261a76 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:16.790929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ffdbd-6a01-45b3-a846-971042feb093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:16.791628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ffdbd-6a01-45b3-a846-971042feb093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.791907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ffdbd-6a01-45b3-a846-971042feb093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.792179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ffdbd-6a01-45b3-a846-971042feb093 admin admin] PATH_INFO: `/v3/projects/d247888e99ff42d4871b4a3ba24b076d/users/6f9b6527173a4d8383dacdc21a67f5cf/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.808606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ffdbd-6a01-45b3-a846-971042feb093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d247888e99ff42d4871b4a3ba24b076d, user_id=6f9b6527173a4d8383dacdc21a67f5cf, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:16.811286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ffdbd-6a01-45b3-a846-971042feb093 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:16.825639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1030/2018] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:16 2022] PUT /identity/v3/projects/d247888e99ff42d4871b4a3ba24b076d/users/6f9b6527173a4d8383dacdc21a67f5cf/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:16.837105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3e90c-2e62-48b3-bb2d-bf13f2e0ea89 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:16.837432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3e90c-2e62-48b3-bb2d-bf13f2e0ea89 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:16.837716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3e90c-2e62-48b3-bb2d-bf13f2e0ea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:16.867255 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-cfc3e90c-2e62-48b3-bb2d-bf13f2e0ea89 None None] MFA Rules not processed for user `6f9b6527173a4d8383dacdc21a67f5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:16.868691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cfc3e90c-2e62-48b3-bb2d-bf13f2e0ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:16.940905 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 989/2019] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:16.946012 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ffdbd-6a01-45b3-a846-971042feb093 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:16.996976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:17.015304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ffdbd-6a01-45b3-a846-971042feb093 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:17.020495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a02e9-faa8-4086-88aa-448f1c42813a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:17.021124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a02e9-faa8-4086-88aa-448f1c42813a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.021449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a02e9-faa8-4086-88aa-448f1c42813a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.021724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a02e9-faa8-4086-88aa-448f1c42813a admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.022407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a02e9-faa8-4086-88aa-448f1c42813a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:17.024353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a02e9-faa8-4086-88aa-448f1c42813a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:17.053800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1031/2020] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:16 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:17.057601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3e90c-2e62-48b3-bb2d-bf13f2e0ea89 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:17.071686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3e90c-2e62-48b3-bb2d-bf13f2e0ea89 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:17.139323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b93fa-e2cc-4574-93de-b908bd260254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:17.139756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b93fa-e2cc-4574-93de-b908bd260254 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.140041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b93fa-e2cc-4574-93de-b908bd260254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.140408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b93fa-e2cc-4574-93de-b908bd260254 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.141106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b93fa-e2cc-4574-93de-b908bd260254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:17.142966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b93fa-e2cc-4574-93de-b908bd260254 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:17.167316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 990/2021] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:17 2022] POST /identity/v3/users => generated 301 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:17.170997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a02e9-faa8-4086-88aa-448f1c42813a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:17.183398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a02e9-faa8-4086-88aa-448f1c42813a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:17.188066 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47aa06-a2bc-4092-9f78-84e6327ac6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:17.188683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47aa06-a2bc-4092-9f78-84e6327ac6cc admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.188964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47aa06-a2bc-4092-9f78-84e6327ac6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.189235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47aa06-a2bc-4092-9f78-84e6327ac6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.189963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47aa06-a2bc-4092-9f78-84e6327ac6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:17.192656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47aa06-a2bc-4092-9f78-84e6327ac6cc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:17.203695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1032/2022] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:17.208661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b93fa-e2cc-4574-93de-b908bd260254 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:17.220872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b93fa-e2cc-4574-93de-b908bd260254 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:17.226584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c508d-45f2-4acd-812b-af4982a77058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:17.227259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c508d-45f2-4acd-812b-af4982a77058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.227545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c508d-45f2-4acd-812b-af4982a77058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.227825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c508d-45f2-4acd-812b-af4982a77058 admin admin] PATH_INFO: `/v3/projects/c47a7c0b907e4538a2d9497b6ffa9206/users/f0c70908be04446f8922a52a43f85d22/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.243696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c508d-45f2-4acd-812b-af4982a77058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47a7c0b907e4538a2d9497b6ffa9206, user_id=f0c70908be04446f8922a52a43f85d22, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:17.246583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c508d-45f2-4acd-812b-af4982a77058 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:17.259095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 991/2023] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:17 2022] PUT /identity/v3/projects/c47a7c0b907e4538a2d9497b6ffa9206/users/f0c70908be04446f8922a52a43f85d22/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:17.263283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47aa06-a2bc-4092-9f78-84e6327ac6cc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:17.273804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47aa06-a2bc-4092-9f78-84e6327ac6cc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:17.330343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914a8c5-8de1-4f1b-bf7e-14b14968c107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:17.331056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914a8c5-8de1-4f1b-bf7e-14b14968c107 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.331338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914a8c5-8de1-4f1b-bf7e-14b14968c107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.331610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914a8c5-8de1-4f1b-bf7e-14b14968c107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.332294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914a8c5-8de1-4f1b-bf7e-14b14968c107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:17.333904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914a8c5-8de1-4f1b-bf7e-14b14968c107 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:17.343427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1033/2024] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:17.348496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c508d-45f2-4acd-812b-af4982a77058 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:17.359403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c508d-45f2-4acd-812b-af4982a77058 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:17.364496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:17.365169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.365487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.365762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] PATH_INFO: `/v3/projects/c47a7c0b907e4538a2d9497b6ffa9206/users/f0c70908be04446f8922a52a43f85d22/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.367767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47a7c0b907e4538a2d9497b6ffa9206, user_id=f0c70908be04446f8922a52a43f85d22, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:17.370436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:17.377226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:17.395005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 992/2025] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:17 2022] PUT /identity/v3/projects/c47a7c0b907e4538a2d9497b6ffa9206/users/f0c70908be04446f8922a52a43f85d22/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:17.396936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b892-7deb-4fb7-b7a4-5f9227f540f7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.397227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b892-7deb-4fb7-b7a4-5f9227f540f7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.397608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b892-7deb-4fb7-b7a4-5f9227f540f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.426590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1028b892-7deb-4fb7-b7a4-5f9227f540f7 None None] MFA Rules not processed for user `f0c70908be04446f8922a52a43f85d22`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:17.427810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1028b892-7deb-4fb7-b7a4-5f9227f540f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:17.513604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1034/2026] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:17.525749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:17.527479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:17.586667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859131f-8b4f-4828-b50a-0160ef6881d9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:17.590860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9476c-8b90-4c4d-b545-fff958229374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:17.591526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9476c-8b90-4c4d-b545-fff958229374 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.591811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9476c-8b90-4c4d-b545-fff958229374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.592089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9476c-8b90-4c4d-b545-fff958229374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.603047 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9476c-8b90-4c4d-b545-fff958229374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:17.604482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9476c-8b90-4c4d-b545-fff958229374 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:17.614691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 993/2027] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:17.753936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028b892-7deb-4fb7-b7a4-5f9227f540f7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:17.800356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:17.820560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028b892-7deb-4fb7-b7a4-5f9227f540f7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:17.825413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:17.826052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.826330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.826601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] PATH_INFO: `/v3/users/6f9b6527173a4d8383dacdc21a67f5cf` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.827816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9b6527173a4d8383dacdc21a67f5cf)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:17.829800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:17.891694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9b6527173a4d8383dacdc21a67f5cf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:17.900975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9b6527173a4d8383dacdc21a67f5cf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:17.918289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9b6527173a4d8383dacdc21a67f5cf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:17.919157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9b6527173a4d8383dacdc21a67f5cf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:17.933228 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9b6527173a4d8383dacdc21a67f5cf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:17.939585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1035/2028] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:17 2022] DELETE /identity/v3/users/6f9b6527173a4d8383dacdc21a67f5cf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:17.945064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c81ff-5094-476f-b60a-7872d3b8928a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:17.945470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c81ff-5094-476f-b60a-7872d3b8928a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:17.945852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c81ff-5094-476f-b60a-7872d3b8928a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:17.974162 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e19c81ff-5094-476f-b60a-7872d3b8928a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:17.975632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e19c81ff-5094-476f-b60a-7872d3b8928a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:18.098234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 994/2029] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:18.109735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:18.111686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:18.131891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c81ff-5094-476f-b60a-7872d3b8928a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:18.134943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e19c81ff-5094-476f-b60a-7872d3b8928a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:18.186675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffd882-fc3b-4c0b-a108-0d22c1db6cdc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:18.192737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:18.193652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:18.194608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:18.194748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:18.207913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:18.209727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:18.209862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c81ff-5094-476f-b60a-7872d3b8928a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:18.220096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:18.220766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:18.222467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:18.222761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:18.224300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1036/2030] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:18.225103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:18.318270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:18.349397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:18.351048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:18.364665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 995/2031] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:18.821042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:18.822765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:18.839032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c31c86-0004-48c9-a394-9d70c95df909 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:18.844161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:18.844817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:18.845101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:18.845445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] PATH_INFO: `/v3/projects/d247888e99ff42d4871b4a3ba24b076d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:18.847841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d247888e99ff42d4871b4a3ba24b076d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:18.849570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:18.904803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd247888e99ff42d4871b4a3ba24b076d', 'request_id': 'req-425a206a-7657-4c7d-8f36-17a5796de2a3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:18.912607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd247888e99ff42d4871b4a3ba24b076d', 'request_id': 'req-425a206a-7657-4c7d-8f36-17a5796de2a3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:18.914572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:18.932777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd247888e99ff42d4871b4a3ba24b076d', 'request_id': 'req-425a206a-7657-4c7d-8f36-17a5796de2a3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:18.948816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] The token cache is being invalidate because project d247888e99ff42d4871b4a3ba24b076d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:18.949110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:18.949813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:18.951345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1037/2032] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:18 2022] DELETE /identity/v3/projects/d247888e99ff42d4871b4a3ba24b076d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:18.955347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:18.965857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bab202-9a2e-4abf-83c6-8a6db56d26a7 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:19.023974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:19.024666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:19.024949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:19.025224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] PATH_INFO: `/v3/users/f0c70908be04446f8922a52a43f85d22` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:19.026848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c70908be04446f8922a52a43f85d22)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:19.028842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:19.082126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c70908be04446f8922a52a43f85d22'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.090983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c70908be04446f8922a52a43f85d22'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.091674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c70908be04446f8922a52a43f85d22'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.106580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c70908be04446f8922a52a43f85d22'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.107346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-975ea0ef-0876-480d-b87a-28005bec7ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c70908be04446f8922a52a43f85d22'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.120760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 996/2033] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:18 2022] DELETE /identity/v3/users/f0c70908be04446f8922a52a43f85d22 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:19.602095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:19.603566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:19.618107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a206a-7657-4c7d-8f36-17a5796de2a3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:19.692136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:19.692800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:19.693079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:19.693389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] PATH_INFO: `/v3/projects/c47a7c0b907e4538a2d9497b6ffa9206` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:19.694849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c47a7c0b907e4538a2d9497b6ffa9206)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:19.696790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:19.749819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c47a7c0b907e4538a2d9497b6ffa9206', 'request_id': 'req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.756455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c47a7c0b907e4538a2d9497b6ffa9206', 'request_id': 'req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.763696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c47a7c0b907e4538a2d9497b6ffa9206', 'request_id': 'req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.775887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] The token cache is being invalidate because project c47a7c0b907e4538a2d9497b6ffa9206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:19.776120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.776777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7e7cc1c9-b2c6-4ae6-90e8-71f87a752433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:19.778400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1038/2034] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:19 2022] DELETE /identity/v3/projects/c47a7c0b907e4538a2d9497b6ffa9206 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:19.783199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fa306-f363-4c16-9058-8c3c5dc19897 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:19.783497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fa306-f363-4c16-9058-8c3c5dc19897 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:19.783830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fa306-f363-4c16-9058-8c3c5dc19897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:19.807660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1b9fa306-f363-4c16-9058-8c3c5dc19897 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:19.808897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1b9fa306-f363-4c16-9058-8c3c5dc19897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:19.874778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 997/2035] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:19.884997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9fef6-1626-4ffb-b237-62a9dbed129d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:19.885399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9fef6-1626-4ffb-b237-62a9dbed129d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:19.885919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9fef6-1626-4ffb-b237-62a9dbed129d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:19.915417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-58e9fef6-1626-4ffb-b237-62a9dbed129d None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:19.919243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-58e9fef6-1626-4ffb-b237-62a9dbed129d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:19.929827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1039/2036] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:19.934239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fa306-f363-4c16-9058-8c3c5dc19897 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:19.945707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fa306-f363-4c16-9058-8c3c5dc19897 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:19.951209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:19.951840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:19.952128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:19.952412 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:19.953277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:19.953634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:19.955242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:19.964932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:19.972621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 998/2037] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:19.977220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9fef6-1626-4ffb-b237-62a9dbed129d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:19.988330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9fef6-1626-4ffb-b237-62a9dbed129d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:19.997106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82a178-9b4b-4d61-a6fb-bac78884273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:19.997968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82a178-9b4b-4d61-a6fb-bac78884273d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:19.998439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82a178-9b4b-4d61-a6fb-bac78884273d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:19.998583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82a178-9b4b-4d61-a6fb-bac78884273d admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:19.999330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82a178-9b4b-4d61-a6fb-bac78884273d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.001196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82a178-9b4b-4d61-a6fb-bac78884273d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.035294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1040/2038] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:19 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:20.039329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.050137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85d769-c09b-4088-82b2-b410626f81ac admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.117086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3407b8d-9a98-4e61-89d7-98aefb0217df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.117828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3407b8d-9a98-4e61-89d7-98aefb0217df admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.118182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3407b8d-9a98-4e61-89d7-98aefb0217df admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.118548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3407b8d-9a98-4e61-89d7-98aefb0217df admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.119290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3407b8d-9a98-4e61-89d7-98aefb0217df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.121268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3407b8d-9a98-4e61-89d7-98aefb0217df admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.144541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 999/2039] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:20 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:20.148262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82a178-9b4b-4d61-a6fb-bac78884273d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.159197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82a178-9b4b-4d61-a6fb-bac78884273d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.164077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f22b8-c8fc-424a-b281-e7841d11e217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.164800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f22b8-c8fc-424a-b281-e7841d11e217 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.165028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f22b8-c8fc-424a-b281-e7841d11e217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.165314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f22b8-c8fc-424a-b281-e7841d11e217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.166045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f22b8-c8fc-424a-b281-e7841d11e217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.167635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f22b8-c8fc-424a-b281-e7841d11e217 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.181138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1041/2040] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:20.182284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3407b8d-9a98-4e61-89d7-98aefb0217df admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.193450 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3407b8d-9a98-4e61-89d7-98aefb0217df admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.198405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1f45b-e671-46a8-859a-2fd1c860d38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.199024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f45b-e671-46a8-859a-2fd1c860d38e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.199312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f45b-e671-46a8-859a-2fd1c860d38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.199597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f45b-e671-46a8-859a-2fd1c860d38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.200286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1f45b-e671-46a8-859a-2fd1c860d38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.202011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1f45b-e671-46a8-859a-2fd1c860d38e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.216535 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1000/2041] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:20.217893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f22b8-c8fc-424a-b281-e7841d11e217 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.228484 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f22b8-c8fc-424a-b281-e7841d11e217 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.233025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de022e-a28a-494a-b303-f9bcb0d97ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.233710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de022e-a28a-494a-b303-f9bcb0d97ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.233992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de022e-a28a-494a-b303-f9bcb0d97ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.234265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de022e-a28a-494a-b303-f9bcb0d97ed0 admin admin] PATH_INFO: `/v3/projects/9dc15c1256b74247a59ca7a1ff874600/users/d06b3c77b49044aeb8ecd45b2ba9be9a/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.250164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de022e-a28a-494a-b303-f9bcb0d97ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc15c1256b74247a59ca7a1ff874600, user_id=d06b3c77b49044aeb8ecd45b2ba9be9a, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.253834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de022e-a28a-494a-b303-f9bcb0d97ed0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.266391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1042/2042] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:20 2022] PUT /identity/v3/projects/9dc15c1256b74247a59ca7a1ff874600/users/d06b3c77b49044aeb8ecd45b2ba9be9a/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:20.278306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03caf1-28ab-4d42-ae9b-e834aec222a5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.278534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03caf1-28ab-4d42-ae9b-e834aec222a5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.278814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03caf1-28ab-4d42-ae9b-e834aec222a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.307649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-da03caf1-28ab-4d42-ae9b-e834aec222a5 None None] MFA Rules not processed for user `d06b3c77b49044aeb8ecd45b2ba9be9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:20.308892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-da03caf1-28ab-4d42-ae9b-e834aec222a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:20.374533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1001/2043] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:20.379312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de022e-a28a-494a-b303-f9bcb0d97ed0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.392281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:20.447942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de022e-a28a-494a-b303-f9bcb0d97ed0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.455400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96896c15-bc76-4ccb-956b-8a2ba1746889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.458285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96896c15-bc76-4ccb-956b-8a2ba1746889 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.458285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96896c15-bc76-4ccb-956b-8a2ba1746889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.458285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96896c15-bc76-4ccb-956b-8a2ba1746889 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.459079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96896c15-bc76-4ccb-956b-8a2ba1746889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.461844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96896c15-bc76-4ccb-956b-8a2ba1746889 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.491292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1043/2044] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:20 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:20.495205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03caf1-28ab-4d42-ae9b-e834aec222a5 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.505903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03caf1-28ab-4d42-ae9b-e834aec222a5 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.566720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f78e4f-08ab-48a9-9ce8-5599a362ec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.567442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f78e4f-08ab-48a9-9ce8-5599a362ec6b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.567731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f78e4f-08ab-48a9-9ce8-5599a362ec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.568013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f78e4f-08ab-48a9-9ce8-5599a362ec6b admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.568735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f78e4f-08ab-48a9-9ce8-5599a362ec6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.570978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f78e4f-08ab-48a9-9ce8-5599a362ec6b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.583842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:20.599086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1002/2045] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:20 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:20.600372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96896c15-bc76-4ccb-956b-8a2ba1746889 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.611534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96896c15-bc76-4ccb-956b-8a2ba1746889 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.616605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429b845-8ee1-428f-bb05-96f49801578f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.617220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429b845-8ee1-428f-bb05-96f49801578f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.617540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429b845-8ee1-428f-bb05-96f49801578f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.617834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429b845-8ee1-428f-bb05-96f49801578f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.618532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429b845-8ee1-428f-bb05-96f49801578f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.620115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429b845-8ee1-428f-bb05-96f49801578f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.630635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1044/2046] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:20.634706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f78e4f-08ab-48a9-9ce8-5599a362ec6b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.646661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f78e4f-08ab-48a9-9ce8-5599a362ec6b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.651466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4e622-f45f-4113-aae8-fd83af2a428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.652248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e622-f45f-4113-aae8-fd83af2a428e admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.652537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e622-f45f-4113-aae8-fd83af2a428e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.652816 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e622-f45f-4113-aae8-fd83af2a428e admin admin] PATH_INFO: `/v3/projects/0ebaf36af6f14269bd100af461aadddd/users/a758c80725b64983a577c5424e135837/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.668736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4e622-f45f-4113-aae8-fd83af2a428e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebaf36af6f14269bd100af461aadddd, user_id=a758c80725b64983a577c5424e135837, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.671456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4e622-f45f-4113-aae8-fd83af2a428e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.686199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1003/2047] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:20 2022] PUT /identity/v3/projects/0ebaf36af6f14269bd100af461aadddd/users/a758c80725b64983a577c5424e135837/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:20.690262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429b845-8ee1-428f-bb05-96f49801578f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.700635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429b845-8ee1-428f-bb05-96f49801578f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.756742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd27b3a-1b9f-4a34-a76b-849fdabf13bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.757412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd27b3a-1b9f-4a34-a76b-849fdabf13bf admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.757699 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd27b3a-1b9f-4a34-a76b-849fdabf13bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.757977 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd27b3a-1b9f-4a34-a76b-849fdabf13bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.758676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd27b3a-1b9f-4a34-a76b-849fdabf13bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.760263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd27b3a-1b9f-4a34-a76b-849fdabf13bf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.771075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1045/2048] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:20.775278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4e622-f45f-4113-aae8-fd83af2a428e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.787512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4e622-f45f-4113-aae8-fd83af2a428e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.793713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.794454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.794722 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.794998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] PATH_INFO: `/v3/projects/0ebaf36af6f14269bd100af461aadddd/users/a758c80725b64983a577c5424e135837/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.797129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebaf36af6f14269bd100af461aadddd, user_id=a758c80725b64983a577c5424e135837, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:20.800563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:20.811091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1004/2049] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:20 2022] PUT /identity/v3/projects/0ebaf36af6f14269bd100af461aadddd/users/a758c80725b64983a577c5424e135837/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:20.822575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1488995-31a4-474e-914d-7937826e01aa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.822993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1488995-31a4-474e-914d-7937826e01aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.823279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1488995-31a4-474e-914d-7937826e01aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:20.851163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e1488995-31a4-474e-914d-7937826e01aa None None] MFA Rules not processed for user `a758c80725b64983a577c5424e135837`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:20.852807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e1488995-31a4-474e-914d-7937826e01aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:20.922576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1046/2050] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:20.933818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:20.935573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:20.994300 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59467b-2ca8-4d67-918d-e4e98fe0f6f8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:20.998687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f3684-eb91-4db4-875a-98d9b38169c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:20.999339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f3684-eb91-4db4-875a-98d9b38169c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:20.999624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f3684-eb91-4db4-875a-98d9b38169c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:20.999894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f3684-eb91-4db4-875a-98d9b38169c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:21.010257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f3684-eb91-4db4-875a-98d9b38169c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:21.011664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f3684-eb91-4db4-875a-98d9b38169c0 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:21.021951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1005/2051] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:21.433878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1488995-31a4-474e-914d-7937826e01aa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:21.443428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:21.498310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1488995-31a4-474e-914d-7937826e01aa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:21.503026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:21.503670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:21.503949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:21.504225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] PATH_INFO: `/v3/users/d06b3c77b49044aeb8ecd45b2ba9be9a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:21.505412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06b3c77b49044aeb8ecd45b2ba9be9a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:21.507331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:21.552798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06b3c77b49044aeb8ecd45b2ba9be9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:21.559747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06b3c77b49044aeb8ecd45b2ba9be9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:21.567684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06b3c77b49044aeb8ecd45b2ba9be9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:21.568332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06b3c77b49044aeb8ecd45b2ba9be9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:21.581507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06b3c77b49044aeb8ecd45b2ba9be9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:21.587367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1047/2052] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:21 2022] DELETE /identity/v3/users/d06b3c77b49044aeb8ecd45b2ba9be9a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:21.591873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff588-fd3d-4514-8a9f-050fd6ca4b7f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:21.592168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff588-fd3d-4514-8a9f-050fd6ca4b7f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:21.592452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff588-fd3d-4514-8a9f-050fd6ca4b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:21.611631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-30aff588-fd3d-4514-8a9f-050fd6ca4b7f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:21.612894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-30aff588-fd3d-4514-8a9f-050fd6ca4b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:21.673565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1006/2053] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:21.683650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:21.685215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:21.756548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43455fd8-5d90-4f16-99a2-470c1f83a3c8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:21.760242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fa516-f4fa-4903-9853-bb0c45eb9aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:21.760861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa516-f4fa-4903-9853-bb0c45eb9aab service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:21.761144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa516-f4fa-4903-9853-bb0c45eb9aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:21.761495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa516-f4fa-4903-9853-bb0c45eb9aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:21.771485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fa516-f4fa-4903-9853-bb0c45eb9aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:21.773121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fa516-f4fa-4903-9853-bb0c45eb9aab service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:21.783188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1048/2054] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:22.062968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aff588-fd3d-4514-8a9f-050fd6ca4b7f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:22.064541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-30aff588-fd3d-4514-8a9f-050fd6ca4b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:22.076899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aff588-fd3d-4514-8a9f-050fd6ca4b7f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:22.082811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:22.083400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:22.083680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:22.083986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] PATH_INFO: `/v3/projects/9dc15c1256b74247a59ca7a1ff874600` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:22.086243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc15c1256b74247a59ca7a1ff874600)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:22.088132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:22.134365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc15c1256b74247a59ca7a1ff874600', 'request_id': 'req-cdbe2966-dbbf-489b-8ec2-524259f18e63'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.141781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc15c1256b74247a59ca7a1ff874600', 'request_id': 'req-cdbe2966-dbbf-489b-8ec2-524259f18e63'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.143668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:22.149890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc15c1256b74247a59ca7a1ff874600', 'request_id': 'req-cdbe2966-dbbf-489b-8ec2-524259f18e63'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.161987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] The token cache is being invalidate because project 9dc15c1256b74247a59ca7a1ff874600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:22.162274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.162907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.164428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1007/2055] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:22 2022] DELETE /identity/v3/projects/9dc15c1256b74247a59ca7a1ff874600 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:22.168635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fa516-f4fa-4903-9853-bb0c45eb9aab service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:22.178289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fa516-f4fa-4903-9853-bb0c45eb9aab service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:22.207497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:22.238879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:22.239572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:22.239858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:22.240135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] PATH_INFO: `/v3/users/a758c80725b64983a577c5424e135837` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:22.241324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a758c80725b64983a577c5424e135837)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:22.243515 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:22.300215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a758c80725b64983a577c5424e135837'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.308286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a758c80725b64983a577c5424e135837'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.322956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a758c80725b64983a577c5424e135837'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.323617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a758c80725b64983a577c5424e135837'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.337439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e46b01e0-885f-4e2a-910d-69b6a6f3ca29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a758c80725b64983a577c5424e135837'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.343419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1049/2056] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:22 2022] DELETE /identity/v3/users/a758c80725b64983a577c5424e135837 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:22.629050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:22.630602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:22.647532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2966-dbbf-489b-8ec2-524259f18e63 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:22.713000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:22.713679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:22.713993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:22.714271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] PATH_INFO: `/v3/projects/0ebaf36af6f14269bd100af461aadddd` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:22.715428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebaf36af6f14269bd100af461aadddd)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:22.717461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:22.763460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebaf36af6f14269bd100af461aadddd', 'request_id': 'req-6f386fbd-be53-4e58-88d2-b970df8a57d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.772024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebaf36af6f14269bd100af461aadddd', 'request_id': 'req-6f386fbd-be53-4e58-88d2-b970df8a57d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.778294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebaf36af6f14269bd100af461aadddd', 'request_id': 'req-6f386fbd-be53-4e58-88d2-b970df8a57d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.791634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] The token cache is being invalidate because project 0ebaf36af6f14269bd100af461aadddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:22.791931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.793358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f386fbd-be53-4e58-88d2-b970df8a57d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:22.795070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1008/2057] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:22 2022] DELETE /identity/v3/projects/0ebaf36af6f14269bd100af461aadddd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:22.799913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73681af4-b701-4960-90fa-087f3a928937 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:22.800386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73681af4-b701-4960-90fa-087f3a928937 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:22.800868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73681af4-b701-4960-90fa-087f3a928937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:22.821857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-73681af4-b701-4960-90fa-087f3a928937 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:22.823145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-73681af4-b701-4960-90fa-087f3a928937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:22.884789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1050/2058] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:22.895135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8f71-8f14-48a1-977e-8fee39a8ea8c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:22.895584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8f71-8f14-48a1-977e-8fee39a8ea8c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:22.895901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8f71-8f14-48a1-977e-8fee39a8ea8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:22.918209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0fea8f71-8f14-48a1-977e-8fee39a8ea8c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:22.919526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0fea8f71-8f14-48a1-977e-8fee39a8ea8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:22.932305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1009/2059] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:22.933369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73681af4-b701-4960-90fa-087f3a928937 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:22.956240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73681af4-b701-4960-90fa-087f3a928937 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:22.961262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:22.961904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:22.962185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:22.962459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:22.963159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:22.963466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:22.965066 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:22.978084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1051/2060] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:38:22.982178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea8f71-8f14-48a1-977e-8fee39a8ea8c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:22.991842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea8f71-8f14-48a1-977e-8fee39a8ea8c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:22.997871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e98cd3-68b5-4b68-a281-69b7b117e545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:22.998497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e98cd3-68b5-4b68-a281-69b7b117e545 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:22.998785 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e98cd3-68b5-4b68-a281-69b7b117e545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:22.999062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e98cd3-68b5-4b68-a281-69b7b117e545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:22.999744 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e98cd3-68b5-4b68-a281-69b7b117e545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:23.001704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e98cd3-68b5-4b68-a281-69b7b117e545 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:23.034565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1010/2061] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:22 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:23.035521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:23.045948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8155918-6c50-46cc-8943-f62c9d57834a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:23.089528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:23.120033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedfa79-ac6c-44b9-a5bc-46ba2e14d4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:23.120728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedfa79-ac6c-44b9-a5bc-46ba2e14d4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.121014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedfa79-ac6c-44b9-a5bc-46ba2e14d4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.121372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedfa79-ac6c-44b9-a5bc-46ba2e14d4cc admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.122122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedfa79-ac6c-44b9-a5bc-46ba2e14d4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:23.124191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedfa79-ac6c-44b9-a5bc-46ba2e14d4cc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:23.147112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1052/2062] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:23 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:23.151326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e98cd3-68b5-4b68-a281-69b7b117e545 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:23.161778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e98cd3-68b5-4b68-a281-69b7b117e545 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:23.166420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349690e4-a7aa-4af7-b38c-2a2dce203d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:23.167031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349690e4-a7aa-4af7-b38c-2a2dce203d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.167317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349690e4-a7aa-4af7-b38c-2a2dce203d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.167594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349690e4-a7aa-4af7-b38c-2a2dce203d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.168288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349690e4-a7aa-4af7-b38c-2a2dce203d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:23.169953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349690e4-a7aa-4af7-b38c-2a2dce203d49 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:23.180043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1011/2063] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:23.184023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedfa79-ac6c-44b9-a5bc-46ba2e14d4cc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:23.195040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedfa79-ac6c-44b9-a5bc-46ba2e14d4cc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:23.199575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989a994-0567-4191-bb43-f454bdafe476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:23.200177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989a994-0567-4191-bb43-f454bdafe476 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.200456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989a994-0567-4191-bb43-f454bdafe476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.200728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989a994-0567-4191-bb43-f454bdafe476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.201446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989a994-0567-4191-bb43-f454bdafe476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:23.203008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989a994-0567-4191-bb43-f454bdafe476 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:23.213441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1053/2064] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:23.217501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349690e4-a7aa-4af7-b38c-2a2dce203d49 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:23.228077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349690e4-a7aa-4af7-b38c-2a2dce203d49 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:23.232858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:23.233550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.233829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.234104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] PATH_INFO: `/v3/projects/894fb0bfdc8c4c8cb77d58c6d88f40bd/users/59ce5e077aab45e1a8129b4a3a698d8c/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.250908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894fb0bfdc8c4c8cb77d58c6d88f40bd, user_id=59ce5e077aab45e1a8129b4a3a698d8c, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:23.254221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:23.266941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1012/2065] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:23 2022] PUT /identity/v3/projects/894fb0bfdc8c4c8cb77d58c6d88f40bd/users/59ce5e077aab45e1a8129b4a3a698d8c/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:23.277967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0caaf-2562-4225-8d10-f3f69fde6eda None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.278254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0caaf-2562-4225-8d10-f3f69fde6eda None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.278526 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0caaf-2562-4225-8d10-f3f69fde6eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.308125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8fb0caaf-2562-4225-8d10-f3f69fde6eda None None] MFA Rules not processed for user `59ce5e077aab45e1a8129b4a3a698d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:23.309411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8fb0caaf-2562-4225-8d10-f3f69fde6eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:23.402208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1054/2066] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:23.408845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:23.411356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:23.428783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:23.478909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bfede-5e7a-405c-b52d-88170c964227 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:23.482682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bb43f-d737-4fa8-86e2-b30ff7ad8308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:23.483408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bb43f-d737-4fa8-86e2-b30ff7ad8308 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.483757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bb43f-d737-4fa8-86e2-b30ff7ad8308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.484112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bb43f-d737-4fa8-86e2-b30ff7ad8308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.494557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bb43f-d737-4fa8-86e2-b30ff7ad8308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:23.496146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bb43f-d737-4fa8-86e2-b30ff7ad8308 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:23.507416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1013/2067] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:23.530649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0caaf-2562-4225-8d10-f3f69fde6eda None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:23.598678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0caaf-2562-4225-8d10-f3f69fde6eda None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:23.605171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:23.606416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.606912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.607375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] PATH_INFO: `/v3/users/59ce5e077aab45e1a8129b4a3a698d8c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.609117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ce5e077aab45e1a8129b4a3a698d8c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:23.611106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:23.660472 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ce5e077aab45e1a8129b4a3a698d8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:23.661593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:23.669137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ce5e077aab45e1a8129b4a3a698d8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:23.677434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ce5e077aab45e1a8129b4a3a698d8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:23.678059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ce5e077aab45e1a8129b4a3a698d8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:23.692354 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ce5e077aab45e1a8129b4a3a698d8c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:23.698509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1055/2068] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:23 2022] DELETE /identity/v3/users/59ce5e077aab45e1a8129b4a3a698d8c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:23.703407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190edac-278c-475f-bb18-afb2329b6f01 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.703708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190edac-278c-475f-bb18-afb2329b6f01 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.703990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190edac-278c-475f-bb18-afb2329b6f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.723211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d190edac-278c-475f-bb18-afb2329b6f01 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:23.724471 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d190edac-278c-475f-bb18-afb2329b6f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:23.789311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1014/2069] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:23.799145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:23.801908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:23.861276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f825e4-049a-4675-939b-b7ed5bbce630 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:23.865690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4b309-f5ad-4631-9209-ebef1259680d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:23.866786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4b309-f5ad-4631-9209-ebef1259680d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:23.866786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4b309-f5ad-4631-9209-ebef1259680d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:23.866982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4b309-f5ad-4631-9209-ebef1259680d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:23.876961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4b309-f5ad-4631-9209-ebef1259680d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:23.879196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4b309-f5ad-4631-9209-ebef1259680d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:23.889880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1056/2070] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:24.177361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190edac-278c-475f-bb18-afb2329b6f01 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.179016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d190edac-278c-475f-bb18-afb2329b6f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:24.207698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190edac-278c-475f-bb18-afb2329b6f01 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.212977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.214366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.214950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.215436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] PATH_INFO: `/v3/projects/894fb0bfdc8c4c8cb77d58c6d88f40bd` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.216829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=894fb0bfdc8c4c8cb77d58c6d88f40bd)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.218936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.278109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894fb0bfdc8c4c8cb77d58c6d88f40bd', 'request_id': 'req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.289254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894fb0bfdc8c4c8cb77d58c6d88f40bd', 'request_id': 'req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.301129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894fb0bfdc8c4c8cb77d58c6d88f40bd', 'request_id': 'req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.313264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] The token cache is being invalidate because project 894fb0bfdc8c4c8cb77d58c6d88f40bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:24.313585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.314232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-b30df8f2-64ca-42c0-8e62-e6bd437cc63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.316257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1015/2071] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:24 2022] DELETE /identity/v3/projects/894fb0bfdc8c4c8cb77d58c6d88f40bd => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:24.321458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d5fa3-496a-4b0b-81fe-c52a129cbc10 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.321750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d5fa3-496a-4b0b-81fe-c52a129cbc10 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.322059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d5fa3-496a-4b0b-81fe-c52a129cbc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.341841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f13d5fa3-496a-4b0b-81fe-c52a129cbc10 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:24.343098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f13d5fa3-496a-4b0b-81fe-c52a129cbc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:24.401806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1057/2072] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:24.409999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41002937-f7ca-41e0-9455-201995ac3222 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.410290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41002937-f7ca-41e0-9455-201995ac3222 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.410563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41002937-f7ca-41e0-9455-201995ac3222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.430499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-41002937-f7ca-41e0-9455-201995ac3222 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:24.431704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-41002937-f7ca-41e0-9455-201995ac3222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:24.442409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1016/2073] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:24.448372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d5fa3-496a-4b0b-81fe-c52a129cbc10 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.460333 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d5fa3-496a-4b0b-81fe-c52a129cbc10 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.464999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.466375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.466375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.466375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.467022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:24.467323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.469009 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.470207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41002937-f7ca-41e0-9455-201995ac3222 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.473223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-41002937-f7ca-41e0-9455-201995ac3222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:24.485519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1058/2074] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:24.486185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41002937-f7ca-41e0-9455-201995ac3222 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.486805 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.490784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.491427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.491706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.492377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] PATH_INFO: `/v3/users/cf15541af8d548cdadca05f522291aa8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.493650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf15541af8d548cdadca05f522291aa8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.495551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.496785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3ce48-7dd2-40bd-b251-79f9592edf55 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.501827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3484f-bc76-4e88-a209-3f03936b0199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.502434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3484f-bc76-4e88-a209-3f03936b0199 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.502713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3484f-bc76-4e88-a209-3f03936b0199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.502988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3484f-bc76-4e88-a209-3f03936b0199 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.503648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3484f-bc76-4e88-a209-3f03936b0199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.505578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3484f-bc76-4e88-a209-3f03936b0199 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.524665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:24.537194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1059/2075] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:24 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:24.541407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3484f-bc76-4e88-a209-3f03936b0199 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.546447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf15541af8d548cdadca05f522291aa8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.554440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf15541af8d548cdadca05f522291aa8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.555117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf15541af8d548cdadca05f522291aa8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.569304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf15541af8d548cdadca05f522291aa8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.570014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0cf4f36-adf5-4e95-a142-45c5f704d1ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf15541af8d548cdadca05f522291aa8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:24.578359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:24.583400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1017/2076] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:24 2022] DELETE /identity/v3/users/cf15541af8d548cdadca05f522291aa8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:24.589327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a335a-2027-437a-afd0-4e429ecf2ce3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.589327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a335a-2027-437a-afd0-4e429ecf2ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.589327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a335a-2027-437a-afd0-4e429ecf2ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.613369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ed1a335a-2027-437a-afd0-4e429ecf2ce3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:24.615469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ed1a335a-2027-437a-afd0-4e429ecf2ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:24.618825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3484f-bc76-4e88-a209-3f03936b0199 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.685947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.686625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.687093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.687502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.688475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.690841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.698336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1018/2077] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:24.705625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a335a-2027-437a-afd0-4e429ecf2ce3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.707372 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ed1a335a-2027-437a-afd0-4e429ecf2ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:24.716225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1060/2078] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:24 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:24.720779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.723829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:24.741968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5439bc0-c464-42d0-9560-c74ad8117517 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.749237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41fdfe-41d3-4e72-9ed8-78ca17cc3861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.749944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41fdfe-41d3-4e72-9ed8-78ca17cc3861 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.750227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41fdfe-41d3-4e72-9ed8-78ca17cc3861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.750803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41fdfe-41d3-4e72-9ed8-78ca17cc3861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.751201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41fdfe-41d3-4e72-9ed8-78ca17cc3861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.752964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41fdfe-41d3-4e72-9ed8-78ca17cc3861 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.769617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1061/2079] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:38:24.770650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41fdfe-41d3-4e72-9ed8-78ca17cc3861 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.781833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41fdfe-41d3-4e72-9ed8-78ca17cc3861 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.784029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a335a-2027-437a-afd0-4e429ecf2ce3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.789695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323979e1-f0e9-49e6-8b03-d94e08d198c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.790243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca751f-8a71-41d0-820d-799c3ad88896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.790484 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323979e1-f0e9-49e6-8b03-d94e08d198c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.790926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323979e1-f0e9-49e6-8b03-d94e08d198c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.791274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323979e1-f0e9-49e6-8b03-d94e08d198c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.791704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca751f-8a71-41d0-820d-799c3ad88896 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.791994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca751f-8a71-41d0-820d-799c3ad88896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.792264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca751f-8a71-41d0-820d-799c3ad88896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.792364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323979e1-f0e9-49e6-8b03-d94e08d198c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.794071 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323979e1-f0e9-49e6-8b03-d94e08d198c0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.805126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca751f-8a71-41d0-820d-799c3ad88896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.807523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca751f-8a71-41d0-820d-799c3ad88896 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.807756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1062/2080] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:24.818157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323979e1-f0e9-49e6-8b03-d94e08d198c0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:24.822025 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1019/2081] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:24.832966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323979e1-f0e9-49e6-8b03-d94e08d198c0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:24.837808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:24.838564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.838780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.839047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] PATH_INFO: `/v3/projects/54ef7526b70e4d2488910d95490a42e5/users/46a6e777856d440b9134eae8a5a792c2/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.855929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ef7526b70e4d2488910d95490a42e5, user_id=46a6e777856d440b9134eae8a5a792c2, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:24.858755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:24.876506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1063/2082] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:24 2022] PUT /identity/v3/projects/54ef7526b70e4d2488910d95490a42e5/users/46a6e777856d440b9134eae8a5a792c2/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:24.888214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed9c67-94fe-44cb-bfc6-e70fd9b4be12 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:24.888505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed9c67-94fe-44cb-bfc6-e70fd9b4be12 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:24.888809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed9c67-94fe-44cb-bfc6-e70fd9b4be12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:24.921860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-36ed9c67-94fe-44cb-bfc6-e70fd9b4be12 None None] MFA Rules not processed for user `46a6e777856d440b9134eae8a5a792c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:24.923070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-36ed9c67-94fe-44cb-bfc6-e70fd9b4be12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:24.958736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:25.003016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1020/2083] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:25.014636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:25.020206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:25.148220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed9c67-94fe-44cb-bfc6-e70fd9b4be12 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:25.149865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-36ed9c67-94fe-44cb-bfc6-e70fd9b4be12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:25.168250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c4834-3e6f-416d-860f-770777fe068c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:25.176483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a94d3-ff7c-4fa3-ba7a-15bde0c9c0c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:25.177059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a94d3-ff7c-4fa3-ba7a-15bde0c9c0c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:25.177423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a94d3-ff7c-4fa3-ba7a-15bde0c9c0c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:25.177644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a94d3-ff7c-4fa3-ba7a-15bde0c9c0c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:25.187337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a94d3-ff7c-4fa3-ba7a-15bde0c9c0c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:25.188836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a94d3-ff7c-4fa3-ba7a-15bde0c9c0c3 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:25.198888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1064/2084] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:25.232915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed9c67-94fe-44cb-bfc6-e70fd9b4be12 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:25.237811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:25.239103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:25.239397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:25.239680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] PATH_INFO: `/v3/projects/e433dc0f008b4e89808cae24d9800a2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:25.240889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e433dc0f008b4e89808cae24d9800a2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:25.243197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:25.296539 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e433dc0f008b4e89808cae24d9800a2d', 'request_id': 'req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.306880 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e433dc0f008b4e89808cae24d9800a2d', 'request_id': 'req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.318624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e433dc0f008b4e89808cae24d9800a2d', 'request_id': 'req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.334626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] The token cache is being invalidate because project e433dc0f008b4e89808cae24d9800a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:25.334999 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.335940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.342834 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1021/2085] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:25 2022] DELETE /identity/v3/projects/e433dc0f008b4e89808cae24d9800a2d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:25.344840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a94d3-ff7c-4fa3-ba7a-15bde0c9c0c3 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:25.370539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a94d3-ff7c-4fa3-ba7a-15bde0c9c0c3 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:25.412003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:25.482401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:25.483090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:25.483447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:25.484280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] PATH_INFO: `/v3/users/9fde6e3126d8419dbca1ee92671ae963` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:25.485665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fde6e3126d8419dbca1ee92671ae963)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:25.488772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:25.568373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fde6e3126d8419dbca1ee92671ae963'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.578000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fde6e3126d8419dbca1ee92671ae963'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.587843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fde6e3126d8419dbca1ee92671ae963'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.588716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fde6e3126d8419dbca1ee92671ae963'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.609339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fde6e3126d8419dbca1ee92671ae963'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:25.619553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1065/2086] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:25 2022] DELETE /identity/v3/users/9fde6e3126d8419dbca1ee92671ae963 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:26.023424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:26.024962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:26.053038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257bda-42e5-4ed1-a3ca-a5cf35eadc09 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:26.132680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:26.133387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:26.133714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:26.134445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] PATH_INFO: `/v3/projects/07fd23a10ca640f4b4cab5f1bedbcbe8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:26.135271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] RBAC: Authorizing `identity:delete_project(project_id=07fd23a10ca640f4b4cab5f1bedbcbe8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:26.137244 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:26.148024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:26.201064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07fd23a10ca640f4b4cab5f1bedbcbe8', 'request_id': 'req-e690f099-79ae-4bff-9e12-78550da6432c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.209637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07fd23a10ca640f4b4cab5f1bedbcbe8', 'request_id': 'req-e690f099-79ae-4bff-9e12-78550da6432c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.215708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07fd23a10ca640f4b4cab5f1bedbcbe8', 'request_id': 'req-e690f099-79ae-4bff-9e12-78550da6432c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.230232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] The token cache is being invalidate because project 07fd23a10ca640f4b4cab5f1bedbcbe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:26.230718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.231505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.233678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1022/2087] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:26 2022] DELETE /identity/v3/projects/07fd23a10ca640f4b4cab5f1bedbcbe8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:26.239035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:26.240559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:26.254359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67712e0f-155e-4544-89c6-0bc1b05a7fdf admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:26.320973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:26.321662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:26.321945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:26.322219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] PATH_INFO: `/v3/users/3db50c076e4041328c33a824b73fb564` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:26.323377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db50c076e4041328c33a824b73fb564)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:26.325274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:26.373886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db50c076e4041328c33a824b73fb564'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.375091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:26.382369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db50c076e4041328c33a824b73fb564'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.390919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db50c076e4041328c33a824b73fb564'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.391648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db50c076e4041328c33a824b73fb564'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.404594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-32213438-d5ea-44eb-b687-ccf3071d0a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db50c076e4041328c33a824b73fb564'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.411092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1066/2088] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:26 2022] DELETE /identity/v3/users/3db50c076e4041328c33a824b73fb564 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:26.729147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:26.731496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:26.759577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690f099-79ae-4bff-9e12-78550da6432c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:26.819911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:26.820685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:26.821091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:26.821414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] PATH_INFO: `/v3/projects/b2a804d5f2d1411b9942b9790092e0d4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:26.822688 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a804d5f2d1411b9942b9790092e0d4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:26.824569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:26.873015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a804d5f2d1411b9942b9790092e0d4', 'request_id': 'req-2229cb75-3e39-4c1d-ac27-cc3722dbe556'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.881214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a804d5f2d1411b9942b9790092e0d4', 'request_id': 'req-2229cb75-3e39-4c1d-ac27-cc3722dbe556'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.887667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a804d5f2d1411b9942b9790092e0d4', 'request_id': 'req-2229cb75-3e39-4c1d-ac27-cc3722dbe556'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.900112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] The token cache is being invalidate because project b2a804d5f2d1411b9942b9790092e0d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:26.900406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.901097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2229cb75-3e39-4c1d-ac27-cc3722dbe556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:26.902659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1023/2089] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:26 2022] DELETE /identity/v3/projects/b2a804d5f2d1411b9942b9790092e0d4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:26.907909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15b68-6bb0-4307-8f8a-e7019fde6b09 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:26.908186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15b68-6bb0-4307-8f8a-e7019fde6b09 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:26.909066 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15b68-6bb0-4307-8f8a-e7019fde6b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:26.933635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5dd15b68-6bb0-4307-8f8a-e7019fde6b09 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:26.934884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5dd15b68-6bb0-4307-8f8a-e7019fde6b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:27.028372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1067/2090] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:27.033278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29bf58-9374-44eb-a4e3-970863e4f148 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.034323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29bf58-9374-44eb-a4e3-970863e4f148 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.034526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29bf58-9374-44eb-a4e3-970863e4f148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.057009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-be29bf58-9374-44eb-a4e3-970863e4f148 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:27.059211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-be29bf58-9374-44eb-a4e3-970863e4f148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:27.074009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1024/2091] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:27.078153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd15b68-6bb0-4307-8f8a-e7019fde6b09 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:27.095918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd15b68-6bb0-4307-8f8a-e7019fde6b09 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:27.103393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:27.104179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.104561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.105080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.105907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:27.106321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:27.108015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:27.122185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1068/2092] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:38:27.126641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29bf58-9374-44eb-a4e3-970863e4f148 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:27.137259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29bf58-9374-44eb-a4e3-970863e4f148 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:27.141971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccabf8-e8d9-445d-b965-c4195fca9535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:27.142604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccabf8-e8d9-445d-b965-c4195fca9535 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.143735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccabf8-e8d9-445d-b965-c4195fca9535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.143982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccabf8-e8d9-445d-b965-c4195fca9535 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.144664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccabf8-e8d9-445d-b965-c4195fca9535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:27.146609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccabf8-e8d9-445d-b965-c4195fca9535 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:27.182782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:27.197153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1025/2093] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:27 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:27.201543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:27.217634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f163201-0591-49fa-8d7a-b141b243d0f7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:27.308289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e63929-4f43-4c7f-928b-b3681a16e6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:27.308999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e63929-4f43-4c7f-928b-b3681a16e6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.309193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e63929-4f43-4c7f-928b-b3681a16e6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.309501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e63929-4f43-4c7f-928b-b3681a16e6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.310323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e63929-4f43-4c7f-928b-b3681a16e6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:27.312229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e63929-4f43-4c7f-928b-b3681a16e6a0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:27.339220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1069/2094] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:27 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:27.340865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccabf8-e8d9-445d-b965-c4195fca9535 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:27.351565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccabf8-e8d9-445d-b965-c4195fca9535 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:27.356386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90d53d-92af-4884-94bd-d7c689feac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:27.357007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90d53d-92af-4884-94bd-d7c689feac0f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.357328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90d53d-92af-4884-94bd-d7c689feac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.357740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90d53d-92af-4884-94bd-d7c689feac0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.358618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90d53d-92af-4884-94bd-d7c689feac0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:27.360274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90d53d-92af-4884-94bd-d7c689feac0f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:27.371683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1026/2095] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:27.375687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e63929-4f43-4c7f-928b-b3681a16e6a0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:27.386985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e63929-4f43-4c7f-928b-b3681a16e6a0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:27.391663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf417c82-4bfa-4e57-bef0-35a608049657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:27.392276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf417c82-4bfa-4e57-bef0-35a608049657 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.392562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf417c82-4bfa-4e57-bef0-35a608049657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.392864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf417c82-4bfa-4e57-bef0-35a608049657 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.393681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf417c82-4bfa-4e57-bef0-35a608049657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:27.395340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf417c82-4bfa-4e57-bef0-35a608049657 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:27.405711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1070/2096] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:27.410172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90d53d-92af-4884-94bd-d7c689feac0f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:27.420903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90d53d-92af-4884-94bd-d7c689feac0f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:27.426334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:27.427046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.427437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.427721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] PATH_INFO: `/v3/projects/3217927667a344baa8e8b870a31ba69f/users/ccf0ecf96d0e45668b32cc4af2cee5ed/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.445959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3217927667a344baa8e8b870a31ba69f, user_id=ccf0ecf96d0e45668b32cc4af2cee5ed, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:27.448628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:27.462982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1027/2097] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:27 2022] PUT /identity/v3/projects/3217927667a344baa8e8b870a31ba69f/users/ccf0ecf96d0e45668b32cc4af2cee5ed/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:27.474355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a3fc-d87e-44ac-b0df-b0a0b1d7d6ea None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.474644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a3fc-d87e-44ac-b0df-b0a0b1d7d6ea None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.474923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a3fc-d87e-44ac-b0df-b0a0b1d7d6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.504541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e160a3fc-d87e-44ac-b0df-b0a0b1d7d6ea None None] MFA Rules not processed for user `ccf0ecf96d0e45668b32cc4af2cee5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:27.505823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e160a3fc-d87e-44ac-b0df-b0a0b1d7d6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:27.524652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:27.577623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1071/2098] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:27.588940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:27.590730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:27.663170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788d37b-726b-4d65-88e8-2f06487badd6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:27.667207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1592d-df46-4d0b-b036-0aa2c12d74cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:27.667846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1592d-df46-4d0b-b036-0aa2c12d74cc service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:27.668141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1592d-df46-4d0b-b036-0aa2c12d74cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:27.668403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1592d-df46-4d0b-b036-0aa2c12d74cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:27.679907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1592d-df46-4d0b-b036-0aa2c12d74cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:27.681405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1592d-df46-4d0b-b036-0aa2c12d74cc service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:27.692543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1028/2099] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:37.574774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=18 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:37.687342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:39.833234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160a3fc-d87e-44ac-b0df-b0a0b1d7d6ea None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:39.896165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160a3fc-d87e-44ac-b0df-b0a0b1d7d6ea None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:39.900640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:39.901278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:39.901597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:39.901873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] PATH_INFO: `/v3/users/ccf0ecf96d0e45668b32cc4af2cee5ed` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:39.903024 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf0ecf96d0e45668b32cc4af2cee5ed)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:39.904938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:39.951213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf0ecf96d0e45668b32cc4af2cee5ed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:39.959487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf0ecf96d0e45668b32cc4af2cee5ed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:39.966816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf0ecf96d0e45668b32cc4af2cee5ed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:39.967454 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf0ecf96d0e45668b32cc4af2cee5ed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:39.979741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf0ecf96d0e45668b32cc4af2cee5ed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:39.985548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1072/2100] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:39 2022] DELETE /identity/v3/users/ccf0ecf96d0e45668b32cc4af2cee5ed => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:39.991196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6284c-308f-4edd-b78e-dceb47bed584 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:39.991673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6284c-308f-4edd-b78e-dceb47bed584 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:39.991892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6284c-308f-4edd-b78e-dceb47bed584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:40.013858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5fc6284c-308f-4edd-b78e-dceb47bed584 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:40.015099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5fc6284c-308f-4edd-b78e-dceb47bed584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:40.078457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1029/2101] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:40.089711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:40.091256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:40.172143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200636cf-1b18-4b41-8121-b154f0a9e4f5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:40.178826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9420e-eec9-4e4f-a4cf-7b5d86d9535e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:40.179627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9420e-eec9-4e4f-a4cf-7b5d86d9535e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:40.180131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9420e-eec9-4e4f-a4cf-7b5d86d9535e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:40.180397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9420e-eec9-4e4f-a4cf-7b5d86d9535e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:40.190782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9420e-eec9-4e4f-a4cf-7b5d86d9535e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:40.192565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9420e-eec9-4e4f-a4cf-7b5d86d9535e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:40.202855 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1073/2102] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:40.571111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6284c-308f-4edd-b78e-dceb47bed584 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:40.572554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5fc6284c-308f-4edd-b78e-dceb47bed584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:40.585911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6284c-308f-4edd-b78e-dceb47bed584 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:40.591570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:40.592304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:40.592709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:40.593073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] PATH_INFO: `/v3/projects/3217927667a344baa8e8b870a31ba69f` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:40.594356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3217927667a344baa8e8b870a31ba69f)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:40.596573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:40.646197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3217927667a344baa8e8b870a31ba69f', 'request_id': 'req-d4058878-e2f7-4a75-8a13-d82f4a474764'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:40.654780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3217927667a344baa8e8b870a31ba69f', 'request_id': 'req-d4058878-e2f7-4a75-8a13-d82f4a474764'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:40.661079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3217927667a344baa8e8b870a31ba69f', 'request_id': 'req-d4058878-e2f7-4a75-8a13-d82f4a474764'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:40.672467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] The token cache is being invalidate because project 3217927667a344baa8e8b870a31ba69f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:40.672757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:40.673420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d4058878-e2f7-4a75-8a13-d82f4a474764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:40.675019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1030/2103] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:40 2022] DELETE /identity/v3/projects/3217927667a344baa8e8b870a31ba69f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:40.681697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4465012-cf9f-44ce-92da-e407a2768094 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:40.681992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4465012-cf9f-44ce-92da-e407a2768094 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:40.682295 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4465012-cf9f-44ce-92da-e407a2768094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:40.702109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e4465012-cf9f-44ce-92da-e407a2768094 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:40.703314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e4465012-cf9f-44ce-92da-e407a2768094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:40.728952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:40.819729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1074/2104] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:40.827363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d763a-d521-41a2-b67d-2940d16297b2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:40.827777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d763a-d521-41a2-b67d-2940d16297b2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:40.828283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d763a-d521-41a2-b67d-2940d16297b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:40.853944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ea9d763a-d521-41a2-b67d-2940d16297b2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:40.855194 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ea9d763a-d521-41a2-b67d-2940d16297b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:40.865568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1031/2105] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:40.871180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4465012-cf9f-44ce-92da-e407a2768094 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:40.884912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4465012-cf9f-44ce-92da-e407a2768094 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:40.890179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:40.890919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:40.891274 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:40.891553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:40.892257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:40.892565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:40.894230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:40.913983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1075/2106] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:40.918341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d763a-d521-41a2-b67d-2940d16297b2 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:40.928614 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d763a-d521-41a2-b67d-2940d16297b2 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:40.934003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef66e-eb52-473f-bed5-0742fa13c208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:40.934481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef66e-eb52-473f-bed5-0742fa13c208 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:40.934765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef66e-eb52-473f-bed5-0742fa13c208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:40.935068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef66e-eb52-473f-bed5-0742fa13c208 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:40.935752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ef66e-eb52-473f-bed5-0742fa13c208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:40.937673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ef66e-eb52-473f-bed5-0742fa13c208 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:40.973147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1032/2107] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:40 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:40.975666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:40.993884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988407a7-6405-4a18-991f-abefc5f0ab43 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:41.074284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bb6d3-ce1b-4d0d-92f3-16dc2158f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:41.074849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb6d3-ce1b-4d0d-92f3-16dc2158f678 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.075134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb6d3-ce1b-4d0d-92f3-16dc2158f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.075412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb6d3-ce1b-4d0d-92f3-16dc2158f678 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.076081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bb6d3-ce1b-4d0d-92f3-16dc2158f678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:41.078026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bb6d3-ce1b-4d0d-92f3-16dc2158f678 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:41.100867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1076/2108] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:40 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:41.104994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef66e-eb52-473f-bed5-0742fa13c208 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:41.116567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ef66e-eb52-473f-bed5-0742fa13c208 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:41.121341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7b06a-9ffc-48a2-8a90-f9ec8d6981c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:41.122011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7b06a-9ffc-48a2-8a90-f9ec8d6981c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.122293 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7b06a-9ffc-48a2-8a90-f9ec8d6981c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.122572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7b06a-9ffc-48a2-8a90-f9ec8d6981c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.123278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7b06a-9ffc-48a2-8a90-f9ec8d6981c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:41.124868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7b06a-9ffc-48a2-8a90-f9ec8d6981c8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:41.135353 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1033/2109] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:41.139259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bb6d3-ce1b-4d0d-92f3-16dc2158f678 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:41.149190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bb6d3-ce1b-4d0d-92f3-16dc2158f678 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:41.153795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:41.154379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.154653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.154924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.155604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:41.157215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:41.168414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1077/2110] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:41.181221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7b06a-9ffc-48a2-8a90-f9ec8d6981c8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:41.191653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7b06a-9ffc-48a2-8a90-f9ec8d6981c8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:41.196835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a76b9d-3809-4822-8065-1ecc21eb0d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:41.197531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a76b9d-3809-4822-8065-1ecc21eb0d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.197815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a76b9d-3809-4822-8065-1ecc21eb0d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.198092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a76b9d-3809-4822-8065-1ecc21eb0d6f admin admin] PATH_INFO: `/v3/projects/f16eee0fb716452e8c760a27b5724d91/users/13ca4be686f84c279f87016c20129833/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.213384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a76b9d-3809-4822-8065-1ecc21eb0d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16eee0fb716452e8c760a27b5724d91, user_id=13ca4be686f84c279f87016c20129833, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:41.216314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a76b9d-3809-4822-8065-1ecc21eb0d6f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:41.216455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:41.218094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:41.231138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1034/2111] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:41 2022] PUT /identity/v3/projects/f16eee0fb716452e8c760a27b5724d91/users/13ca4be686f84c279f87016c20129833/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:41.234945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3c611-553d-4cb1-abd9-4bb240b7061a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:41.236241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f2324-3035-4475-9ff5-30cbe69317ee None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.236530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f2324-3035-4475-9ff5-30cbe69317ee None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.236805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f2324-3035-4475-9ff5-30cbe69317ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.258208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f25f2324-3035-4475-9ff5-30cbe69317ee None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:41.259406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f25f2324-3035-4475-9ff5-30cbe69317ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:41.271814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:41.295022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:41.295689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.295962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.296241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] PATH_INFO: `/v3/users/46a6e777856d440b9134eae8a5a792c2` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.297458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a6e777856d440b9134eae8a5a792c2)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:41.299450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:41.323024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1035/2112] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:41.333923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f2324-3035-4475-9ff5-30cbe69317ee None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:41.337635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f25f2324-3035-4475-9ff5-30cbe69317ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:41.343708 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:41.361737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a6e777856d440b9134eae8a5a792c2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:41.370125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a6e777856d440b9134eae8a5a792c2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:41.379677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a6e777856d440b9134eae8a5a792c2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:41.383277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a6e777856d440b9134eae8a5a792c2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:41.400681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-775ebc71-b7b1-46d7-82cb-187fdce678ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a6e777856d440b9134eae8a5a792c2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:41.407013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1078/2113] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:41 2022] DELETE /identity/v3/users/46a6e777856d440b9134eae8a5a792c2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:41.413026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1b6ad-df02-44ac-98b0-f38b3cd6e0e1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.413586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1b6ad-df02-44ac-98b0-f38b3cd6e0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.414467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1b6ad-df02-44ac-98b0-f38b3cd6e0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.415906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f2324-3035-4475-9ff5-30cbe69317ee None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:41.453500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4dc1b6ad-df02-44ac-98b0-f38b3cd6e0e1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:41.453500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4dc1b6ad-df02-44ac-98b0-f38b3cd6e0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:41.483775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:41.484542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.484958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.485336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.488325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:41.536541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1079/2114] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:41.546961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1b6ad-df02-44ac-98b0-f38b3cd6e0e1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:41.548516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4dc1b6ad-df02-44ac-98b0-f38b3cd6e0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:41.562829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1b6ad-df02-44ac-98b0-f38b3cd6e0e1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:41.567777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797aa32-7ee2-4303-8db6-fec59444db1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:41.568380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797aa32-7ee2-4303-8db6-fec59444db1d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.568823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797aa32-7ee2-4303-8db6-fec59444db1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.569483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797aa32-7ee2-4303-8db6-fec59444db1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.580669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797aa32-7ee2-4303-8db6-fec59444db1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:41.580765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:41.581868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797aa32-7ee2-4303-8db6-fec59444db1d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:41.582184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:41.594610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1036/2115] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:41.600470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1080/2116] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:41.935951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:41.937578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:41.954272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70f195-95ac-43a6-a422-100f7672291f service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:41.960027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:41.960763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:41.961572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:41.961572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] PATH_INFO: `/v3/projects/54ef7526b70e4d2488910d95490a42e5` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:41.962767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ef7526b70e4d2488910d95490a42e5)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:41.964957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:41.997129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:42.030072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ef7526b70e4d2488910d95490a42e5', 'request_id': 'req-ad20130e-4f6e-412d-945a-f80f1ddedbed'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:42.042818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ef7526b70e4d2488910d95490a42e5', 'request_id': 'req-ad20130e-4f6e-412d-945a-f80f1ddedbed'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:42.049763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ef7526b70e4d2488910d95490a42e5', 'request_id': 'req-ad20130e-4f6e-412d-945a-f80f1ddedbed'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:42.063840 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] The token cache is being invalidate because project 54ef7526b70e4d2488910d95490a42e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:42.063993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:42.064669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ad20130e-4f6e-412d-945a-f80f1ddedbed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:42.071473 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1037/2117] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:41 2022] DELETE /identity/v3/projects/54ef7526b70e4d2488910d95490a42e5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:42.073195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0d31-d675-44ae-9e40-f3f5cbc9549e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.073687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0d31-d675-44ae-9e40-f3f5cbc9549e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.073879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0d31-d675-44ae-9e40-f3f5cbc9549e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.095642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9b0d0d31-d675-44ae-9e40-f3f5cbc9549e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:42.096853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b0d0d31-d675-44ae-9e40-f3f5cbc9549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:42.156022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1081/2118] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:42.163799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13882ff-9110-403b-813b-d216a73969f4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.164337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13882ff-9110-403b-813b-d216a73969f4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.164337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13882ff-9110-403b-813b-d216a73969f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.186287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f13882ff-9110-403b-813b-d216a73969f4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:42.187645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f13882ff-9110-403b-813b-d216a73969f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:42.196767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1038/2119] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:42.201284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d0d31-d675-44ae-9e40-f3f5cbc9549e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:42.212318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d0d31-d675-44ae-9e40-f3f5cbc9549e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:42.217074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:42.217737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.218083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.218364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.219101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:42.219407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:42.221042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:42.234919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1082/2120] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:42.239625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13882ff-9110-403b-813b-d216a73969f4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:42.249360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13882ff-9110-403b-813b-d216a73969f4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:42.254976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f4b0-2ba0-40e8-a86a-5f094d8a7167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:42.255644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f4b0-2ba0-40e8-a86a-5f094d8a7167 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.255926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f4b0-2ba0-40e8-a86a-5f094d8a7167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.256198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f4b0-2ba0-40e8-a86a-5f094d8a7167 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.256876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01f4b0-2ba0-40e8-a86a-5f094d8a7167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:42.258832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01f4b0-2ba0-40e8-a86a-5f094d8a7167 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:42.291129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1039/2121] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:42 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:42.295408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:42.306714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ecbf-90f2-470d-8edc-4577de64150c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:42.327286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:42.375841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9921c-7fdd-4ee5-bde7-4a34846c9c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:42.375841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9921c-7fdd-4ee5-bde7-4a34846c9c5e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.375841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9921c-7fdd-4ee5-bde7-4a34846c9c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.375841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9921c-7fdd-4ee5-bde7-4a34846c9c5e admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.376880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9921c-7fdd-4ee5-bde7-4a34846c9c5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:42.376880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9921c-7fdd-4ee5-bde7-4a34846c9c5e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:42.397264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1083/2122] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:42 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:42.401492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f4b0-2ba0-40e8-a86a-5f094d8a7167 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:42.412988 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f4b0-2ba0-40e8-a86a-5f094d8a7167 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:42.418537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d98d4d-06c0-4a08-bfed-53c090d3bc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:42.419164 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98d4d-06c0-4a08-bfed-53c090d3bc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.419449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98d4d-06c0-4a08-bfed-53c090d3bc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.419727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98d4d-06c0-4a08-bfed-53c090d3bc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.420435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d98d4d-06c0-4a08-bfed-53c090d3bc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:42.422072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d98d4d-06c0-4a08-bfed-53c090d3bc84 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:42.433691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1040/2123] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:42.441097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9921c-7fdd-4ee5-bde7-4a34846c9c5e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:42.458123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9921c-7fdd-4ee5-bde7-4a34846c9c5e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:42.462983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a89f4-d595-44a5-9c5c-f8d4d30b8295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:42.463689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a89f4-d595-44a5-9c5c-f8d4d30b8295 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.464044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a89f4-d595-44a5-9c5c-f8d4d30b8295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.464581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a89f4-d595-44a5-9c5c-f8d4d30b8295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.465095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a89f4-d595-44a5-9c5c-f8d4d30b8295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:42.466771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a89f4-d595-44a5-9c5c-f8d4d30b8295 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:42.480309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1084/2124] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:42.484201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d98d4d-06c0-4a08-bfed-53c090d3bc84 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:42.510064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d98d4d-06c0-4a08-bfed-53c090d3bc84 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:42.521447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ae080-4283-492a-ad6d-1c62b1c97533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:42.522142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae080-4283-492a-ad6d-1c62b1c97533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.522428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae080-4283-492a-ad6d-1c62b1c97533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.522708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae080-4283-492a-ad6d-1c62b1c97533 admin admin] PATH_INFO: `/v3/projects/5fd781c3511a40cc897d2968d178c1b6/users/4cc02d30c248492c8d138fcc289405b6/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.540501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ae080-4283-492a-ad6d-1c62b1c97533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd781c3511a40cc897d2968d178c1b6, user_id=4cc02d30c248492c8d138fcc289405b6, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:42.543511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ae080-4283-492a-ad6d-1c62b1c97533 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:42.567031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1041/2125] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:42 2022] PUT /identity/v3/projects/5fd781c3511a40cc897d2968d178c1b6/users/4cc02d30c248492c8d138fcc289405b6/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:42.574682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceff6b-7eb2-43b7-a94f-d0bcd54acf7a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.575439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceff6b-7eb2-43b7-a94f-d0bcd54acf7a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.575721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceff6b-7eb2-43b7-a94f-d0bcd54acf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.608567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-aeceff6b-7eb2-43b7-a94f-d0bcd54acf7a None None] MFA Rules not processed for user `4cc02d30c248492c8d138fcc289405b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:42.609918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aeceff6b-7eb2-43b7-a94f-d0bcd54acf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:42.720092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1085/2126] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:42.721231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ae080-4283-492a-ad6d-1c62b1c97533 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:42.849319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ae080-4283-492a-ad6d-1c62b1c97533 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:42.860516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a786-004e-436f-8771-37e9facad5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:42.861158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a786-004e-436f-8771-37e9facad5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:42.863695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a786-004e-436f-8771-37e9facad5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:42.863695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195a786-004e-436f-8771-37e9facad5b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:42.864350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195a786-004e-436f-8771-37e9facad5b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:42.866300 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195a786-004e-436f-8771-37e9facad5b8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:42.920783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1042/2127] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:42 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:42.925056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeceff6b-7eb2-43b7-a94f-d0bcd54acf7a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:42.936751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeceff6b-7eb2-43b7-a94f-d0bcd54acf7a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.005624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87abf6-1991-45b7-a536-9dadf3a62bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.006265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87abf6-1991-45b7-a536-9dadf3a62bcb admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.006542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87abf6-1991-45b7-a536-9dadf3a62bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.007121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87abf6-1991-45b7-a536-9dadf3a62bcb admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.007632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87abf6-1991-45b7-a536-9dadf3a62bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.009596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87abf6-1991-45b7-a536-9dadf3a62bcb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.037638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1086/2128] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:42 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:43.040107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a786-004e-436f-8771-37e9facad5b8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.051224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195a786-004e-436f-8771-37e9facad5b8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.056351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91dbf8a-d259-442c-ac8f-03c6be36810e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.057055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dbf8a-d259-442c-ac8f-03c6be36810e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.057357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dbf8a-d259-442c-ac8f-03c6be36810e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.057634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dbf8a-d259-442c-ac8f-03c6be36810e admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.058327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91dbf8a-d259-442c-ac8f-03c6be36810e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.059943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91dbf8a-d259-442c-ac8f-03c6be36810e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.073189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1043/2129] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:43.076311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87abf6-1991-45b7-a536-9dadf3a62bcb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.096638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87abf6-1991-45b7-a536-9dadf3a62bcb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.102498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814edde-c083-485a-95be-30c55955d67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.103159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814edde-c083-485a-95be-30c55955d67c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.103439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814edde-c083-485a-95be-30c55955d67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.103714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814edde-c083-485a-95be-30c55955d67c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.104419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814edde-c083-485a-95be-30c55955d67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.106527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814edde-c083-485a-95be-30c55955d67c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.117957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1087/2130] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:43.122177 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91dbf8a-d259-442c-ac8f-03c6be36810e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.141570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91dbf8a-d259-442c-ac8f-03c6be36810e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.148466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3594b-fd3b-44f4-bb70-e327357fd610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.149237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3594b-fd3b-44f4-bb70-e327357fd610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.149620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3594b-fd3b-44f4-bb70-e327357fd610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.149903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3594b-fd3b-44f4-bb70-e327357fd610 admin admin] PATH_INFO: `/v3/projects/b54deb60766e42fe93692996af85f9e4/users/920f1b5b4f384838b730db2e7342dd58/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.170760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3594b-fd3b-44f4-bb70-e327357fd610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54deb60766e42fe93692996af85f9e4, user_id=920f1b5b4f384838b730db2e7342dd58, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.173463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3594b-fd3b-44f4-bb70-e327357fd610 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.179349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:43.203351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1044/2131] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:43 2022] PUT /identity/v3/projects/b54deb60766e42fe93692996af85f9e4/users/920f1b5b4f384838b730db2e7342dd58/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:43.206039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ace25-2f7d-4aa6-81b9-10d906ee7c22 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.206968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ace25-2f7d-4aa6-81b9-10d906ee7c22 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.207835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ace25-2f7d-4aa6-81b9-10d906ee7c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.233007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:43.248084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-503ace25-2f7d-4aa6-81b9-10d906ee7c22 None None] MFA Rules not processed for user `920f1b5b4f384838b730db2e7342dd58`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:43.249788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-503ace25-2f7d-4aa6-81b9-10d906ee7c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:43.336936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1088/2132] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:43.341608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3594b-fd3b-44f4-bb70-e327357fd610 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.432363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3594b-fd3b-44f4-bb70-e327357fd610 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.439779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6705b-bb91-40cd-b8e5-85bb70e4c702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.440463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6705b-bb91-40cd-b8e5-85bb70e4c702 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.440741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6705b-bb91-40cd-b8e5-85bb70e4c702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.441740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6705b-bb91-40cd-b8e5-85bb70e4c702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.442470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6705b-bb91-40cd-b8e5-85bb70e4c702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.444656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6705b-bb91-40cd-b8e5-85bb70e4c702 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.479726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1045/2133] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:43 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:38:43.481030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ace25-2f7d-4aa6-81b9-10d906ee7c22 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.494103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ace25-2f7d-4aa6-81b9-10d906ee7c22 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.571278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dadbfd-9209-4ad2-922a-8a397b5a62a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.571913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dadbfd-9209-4ad2-922a-8a397b5a62a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.572198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dadbfd-9209-4ad2-922a-8a397b5a62a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.572472 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dadbfd-9209-4ad2-922a-8a397b5a62a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.573178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dadbfd-9209-4ad2-922a-8a397b5a62a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.575291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dadbfd-9209-4ad2-922a-8a397b5a62a1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.605146 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1089/2134] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:43 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:43.606404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6705b-bb91-40cd-b8e5-85bb70e4c702 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.617807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6705b-bb91-40cd-b8e5-85bb70e4c702 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.623041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2634d66-b4aa-4def-859a-d674ca0ca249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.623731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2634d66-b4aa-4def-859a-d674ca0ca249 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.623954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2634d66-b4aa-4def-859a-d674ca0ca249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.624229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2634d66-b4aa-4def-859a-d674ca0ca249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.624934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2634d66-b4aa-4def-859a-d674ca0ca249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.627772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2634d66-b4aa-4def-859a-d674ca0ca249 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.638750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1046/2135] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:38:43.642984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dadbfd-9209-4ad2-922a-8a397b5a62a1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.655802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dadbfd-9209-4ad2-922a-8a397b5a62a1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.661417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f50cd-7e0f-4235-9a5d-3de0e0a43120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.662173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f50cd-7e0f-4235-9a5d-3de0e0a43120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.662456 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f50cd-7e0f-4235-9a5d-3de0e0a43120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.662728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f50cd-7e0f-4235-9a5d-3de0e0a43120 admin admin] PATH_INFO: `/v3/projects/f2e176938463495bb87ecea601f3687b/users/db042afc07424460b317e784df11505e/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.679940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f50cd-7e0f-4235-9a5d-3de0e0a43120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e176938463495bb87ecea601f3687b, user_id=db042afc07424460b317e784df11505e, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.682649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f50cd-7e0f-4235-9a5d-3de0e0a43120 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.709117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1090/2136] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:43 2022] PUT /identity/v3/projects/f2e176938463495bb87ecea601f3687b/users/db042afc07424460b317e784df11505e/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:43.713200 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2634d66-b4aa-4def-859a-d674ca0ca249 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.724729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2634d66-b4aa-4def-859a-d674ca0ca249 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.812903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8154dab-11b7-4917-813e-53d7680210e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.813902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8154dab-11b7-4917-813e-53d7680210e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.814190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8154dab-11b7-4917-813e-53d7680210e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.814469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8154dab-11b7-4917-813e-53d7680210e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.815186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8154dab-11b7-4917-813e-53d7680210e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.817096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8154dab-11b7-4917-813e-53d7680210e4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.829161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1047/2137] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:43.833946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f50cd-7e0f-4235-9a5d-3de0e0a43120 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:43.848000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f50cd-7e0f-4235-9a5d-3de0e0a43120 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:43.854543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:43.855462 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.855607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.855776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] PATH_INFO: `/v3/projects/f2e176938463495bb87ecea601f3687b/users/db042afc07424460b317e784df11505e/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.858235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e176938463495bb87ecea601f3687b, user_id=db042afc07424460b317e784df11505e, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:43.861167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:43.874047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1091/2138] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:43 2022] PUT /identity/v3/projects/f2e176938463495bb87ecea601f3687b/users/db042afc07424460b317e784df11505e/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:43.884930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369dbb2-517a-4e26-9968-33b968751f4c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:43.885233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369dbb2-517a-4e26-9968-33b968751f4c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:43.885558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369dbb2-517a-4e26-9968-33b968751f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:43.919831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f369dbb2-517a-4e26-9968-33b968751f4c None None] MFA Rules not processed for user `db042afc07424460b317e784df11505e`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:43.921244 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f369dbb2-517a-4e26-9968-33b968751f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:43.997890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:44.020516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1048/2139] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:44.032222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:44.034109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:44.095552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcccf14-3004-4db3-a781-d7afe2d07c99 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:44.100186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dc9b7-a45c-4014-9cb5-d8ed6e33b274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:44.100807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dc9b7-a45c-4014-9cb5-d8ed6e33b274 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:44.101091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dc9b7-a45c-4014-9cb5-d8ed6e33b274 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:44.101400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dc9b7-a45c-4014-9cb5-d8ed6e33b274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:44.105796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:44.173869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569dc9b7-a45c-4014-9cb5-d8ed6e33b274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:44.176188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569dc9b7-a45c-4014-9cb5-d8ed6e33b274 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:44.186530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1092/2140] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:45.920011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369dbb2-517a-4e26-9968-33b968751f4c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:45.921591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f369dbb2-517a-4e26-9968-33b968751f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:45.934496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369dbb2-517a-4e26-9968-33b968751f4c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:45.939397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:45.940219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:45.940643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:45.941007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:46.017902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:46.020081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:46.038817 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1049/2141] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:46.564070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de73b63-ffec-43db-b8f0-03fb0f2577ca None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:46.564665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de73b63-ffec-43db-b8f0-03fb0f2577ca None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:46.564665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de73b63-ffec-43db-b8f0-03fb0f2577ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:46.606830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5de73b63-ffec-43db-b8f0-03fb0f2577ca None None] MFA Rules not processed for user `13ca4be686f84c279f87016c20129833`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:46.608580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5de73b63-ffec-43db-b8f0-03fb0f2577ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:46.698209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1093/2142] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:46.702680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:46.704440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:46.803029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8678e-7f19-4550-96ce-842f1efb8205 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:46.812328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947ec1a-02a7-4ee8-b990-d7b9e81b69f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:46.813102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947ec1a-02a7-4ee8-b990-d7b9e81b69f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:46.813545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947ec1a-02a7-4ee8-b990-d7b9e81b69f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:46.813886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947ec1a-02a7-4ee8-b990-d7b9e81b69f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:46.819111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947ec1a-02a7-4ee8-b990-d7b9e81b69f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:46.823842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947ec1a-02a7-4ee8-b990-d7b9e81b69f8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:46.865460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1050/2143] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:46 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:46.866833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de73b63-ffec-43db-b8f0-03fb0f2577ca None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:46.883851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de73b63-ffec-43db-b8f0-03fb0f2577ca None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:47.011978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702991da-6630-4610-a663-ef133faa8391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:47.012620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702991da-6630-4610-a663-ef133faa8391 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:47.012902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702991da-6630-4610-a663-ef133faa8391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:47.013183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702991da-6630-4610-a663-ef133faa8391 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:47.013905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702991da-6630-4610-a663-ef133faa8391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:47.015838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702991da-6630-4610-a663-ef133faa8391 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:47.043453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1094/2144] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:46 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:47.047684 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947ec1a-02a7-4ee8-b990-d7b9e81b69f8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:47.066054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947ec1a-02a7-4ee8-b990-d7b9e81b69f8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:47.077376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7161e-5e79-449a-9afb-e2865e20a14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:47.078200 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7161e-5e79-449a-9afb-e2865e20a14b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:47.078651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7161e-5e79-449a-9afb-e2865e20a14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:47.078919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7161e-5e79-449a-9afb-e2865e20a14b admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:47.079757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7161e-5e79-449a-9afb-e2865e20a14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:47.082911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7161e-5e79-449a-9afb-e2865e20a14b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:47.105131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702991da-6630-4610-a663-ef133faa8391 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:47.109360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1051/2145] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:47.116031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7161e-5e79-449a-9afb-e2865e20a14b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:47.137093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7161e-5e79-449a-9afb-e2865e20a14b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:47.140198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:47.147564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c38c8-4f2c-4abc-9566-48768d4466b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:47.148783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c38c8-4f2c-4abc-9566-48768d4466b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:47.149140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c38c8-4f2c-4abc-9566-48768d4466b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:47.149626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c38c8-4f2c-4abc-9566-48768d4466b9 admin admin] PATH_INFO: `/v3/projects/a771358cf916419593e60477ec58b5fa/users/f37a9c5000c54dada98baebc43684446/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:47.167890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c38c8-4f2c-4abc-9566-48768d4466b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a771358cf916419593e60477ec58b5fa, user_id=f37a9c5000c54dada98baebc43684446, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:47.170687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c38c8-4f2c-4abc-9566-48768d4466b9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:47.196826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1052/2146] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:47 2022] PUT /identity/v3/projects/a771358cf916419593e60477ec58b5fa/users/f37a9c5000c54dada98baebc43684446/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:47.200845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c38c8-4f2c-4abc-9566-48768d4466b9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:47.214446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702991da-6630-4610-a663-ef133faa8391 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:47.305183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:47.317341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1d0cd-bf4f-4dac-8779-e33ff0fd42df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:47.318279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1d0cd-bf4f-4dac-8779-e33ff0fd42df service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:47.318725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1d0cd-bf4f-4dac-8779-e33ff0fd42df service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:47.319419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1d0cd-bf4f-4dac-8779-e33ff0fd42df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:47.339382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c38c8-4f2c-4abc-9566-48768d4466b9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:47.349196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cf7ae-b7bb-4f2d-ae0f-7a2e146dbca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:47.350136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cf7ae-b7bb-4f2d-ae0f-7a2e146dbca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:47.350580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cf7ae-b7bb-4f2d-ae0f-7a2e146dbca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:47.351012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cf7ae-b7bb-4f2d-ae0f-7a2e146dbca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:47.352018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cf7ae-b7bb-4f2d-ae0f-7a2e146dbca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:47.354068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cf7ae-b7bb-4f2d-ae0f-7a2e146dbca9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:47.369075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1053/2147] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:47.372795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cf7ae-b7bb-4f2d-ae0f-7a2e146dbca9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:47.383889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cf7ae-b7bb-4f2d-ae0f-7a2e146dbca9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:47.388891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:47.389726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:47.390108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:47.390486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] PATH_INFO: `/v3/projects/a771358cf916419593e60477ec58b5fa/users/f37a9c5000c54dada98baebc43684446/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:47.392868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a771358cf916419593e60477ec58b5fa, user_id=f37a9c5000c54dada98baebc43684446, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:47.396814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:47.411821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1054/2148] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:47 2022] PUT /identity/v3/projects/a771358cf916419593e60477ec58b5fa/users/f37a9c5000c54dada98baebc43684446/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:38:47.415401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1d0cd-bf4f-4dac-8779-e33ff0fd42df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:47.416971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1d0cd-bf4f-4dac-8779-e33ff0fd42df service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:47.430004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1095/2149] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:50.711865 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:50.713669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:50.869218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2911ad3-dea4-4790-be04-afd51c9608ec admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:50.876098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:50.876759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:50.877043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:50.877342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] PATH_INFO: `/v3/users/cd039b9a1b7143a2b204e694b3cc7b0d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:50.887752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd039b9a1b7143a2b204e694b3cc7b0d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:50.889773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:51.079076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd039b9a1b7143a2b204e694b3cc7b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:51.100977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd039b9a1b7143a2b204e694b3cc7b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:51.101654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd039b9a1b7143a2b204e694b3cc7b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:51.140613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd039b9a1b7143a2b204e694b3cc7b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:51.141424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd039b9a1b7143a2b204e694b3cc7b0d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:51.182855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1055/2150] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:50 2022] DELETE /identity/v3/users/cd039b9a1b7143a2b204e694b3cc7b0d => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:51.188754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e110-f8b0-4571-8faa-0e445cfb01f5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:51.189361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e110-f8b0-4571-8faa-0e445cfb01f5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:51.189704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e110-f8b0-4571-8faa-0e445cfb01f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:51.260606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1269e110-f8b0-4571-8faa-0e445cfb01f5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:51.262214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1269e110-f8b0-4571-8faa-0e445cfb01f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:51.356542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1096/2151] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:51.390547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:51.392009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:51.488082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:51.531202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5ce6e-0629-4d03-bdc1-f165b8df9184 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:51.559806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334358bc-36a5-4852-828f-67ec2d1af542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:51.562056 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334358bc-36a5-4852-828f-67ec2d1af542 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:51.563916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334358bc-36a5-4852-828f-67ec2d1af542 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:51.564307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334358bc-36a5-4852-828f-67ec2d1af542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:51.599756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334358bc-36a5-4852-828f-67ec2d1af542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:51.602583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334358bc-36a5-4852-828f-67ec2d1af542 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:51.641007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1056/2152] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:52.179651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e110-f8b0-4571-8faa-0e445cfb01f5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:52.182045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1269e110-f8b0-4571-8faa-0e445cfb01f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:52.204176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e110-f8b0-4571-8faa-0e445cfb01f5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:52.211956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:52.214129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:52.214611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:52.215072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] PATH_INFO: `/v3/users/4cc02d30c248492c8d138fcc289405b6` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:52.217038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc02d30c248492c8d138fcc289405b6)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:52.223958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:52.303678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:52.338789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc02d30c248492c8d138fcc289405b6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.358173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc02d30c248492c8d138fcc289405b6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.372303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc02d30c248492c8d138fcc289405b6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.373043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc02d30c248492c8d138fcc289405b6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.402144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc02d30c248492c8d138fcc289405b6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.418646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1097/2153] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:52 2022] DELETE /identity/v3/users/4cc02d30c248492c8d138fcc289405b6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:52.422959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee16558-71fe-4a74-b226-577d17c86404 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:52.423376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee16558-71fe-4a74-b226-577d17c86404 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:52.423522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee16558-71fe-4a74-b226-577d17c86404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:52.464185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6ee16558-71fe-4a74-b226-577d17c86404 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:52.471101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6ee16558-71fe-4a74-b226-577d17c86404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:52.620964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1057/2154] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:52.640830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:52.642526 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:52.662128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815da1c-8d16-4ef3-8e09-46f2b8f2288f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:52.669506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:52.670185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:52.670831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:52.671215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] PATH_INFO: `/v3/projects/b83b7980e2224a3e99d1d6ea37f5bbb6` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:52.672829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83b7980e2224a3e99d1d6ea37f5bbb6)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:52.675022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:52.690398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee16558-71fe-4a74-b226-577d17c86404 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:52.693427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6ee16558-71fe-4a74-b226-577d17c86404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:52.832813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83b7980e2224a3e99d1d6ea37f5bbb6', 'request_id': 'req-904f663d-22de-4046-849b-7cdb8542aaae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.843395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee16558-71fe-4a74-b226-577d17c86404 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:52.847661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83b7980e2224a3e99d1d6ea37f5bbb6', 'request_id': 'req-904f663d-22de-4046-849b-7cdb8542aaae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.861614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83b7980e2224a3e99d1d6ea37f5bbb6', 'request_id': 'req-904f663d-22de-4046-849b-7cdb8542aaae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.902219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] The token cache is being invalidate because project b83b7980e2224a3e99d1d6ea37f5bbb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:52.902560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.903225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:52.904787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1098/2155] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:52 2022] DELETE /identity/v3/projects/b83b7980e2224a3e99d1d6ea37f5bbb6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:52.908767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:52.911478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:52.936053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:52.936863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:52.937239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:52.937699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:52.940607 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:53.018224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f663d-22de-4046-849b-7cdb8542aaae admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:53.046513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:53.047087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:53.047431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:53.047756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] PATH_INFO: `/v3/users/860eea71fb514c1d80f4192067d66512` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:53.053999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] RBAC: Authorizing `identity:delete_user(user_id=860eea71fb514c1d80f4192067d66512)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:53.056591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:53.076451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:53.110602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:53.111801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:53.131117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:53.144870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1058/2156] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:53.174567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860eea71fb514c1d80f4192067d66512'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:53.193099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860eea71fb514c1d80f4192067d66512'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:53.222786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860eea71fb514c1d80f4192067d66512'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:53.224334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860eea71fb514c1d80f4192067d66512'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:53.255200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860eea71fb514c1d80f4192067d66512'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:53.273382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1099/2157] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:52 2022] DELETE /identity/v3/users/860eea71fb514c1d80f4192067d66512 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:53.686601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:53.689069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:53.715477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54080d6d-1d11-406a-b857-6a1f3759bc87 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:53.853325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:53.854814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:53.855330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:53.855819 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] PATH_INFO: `/v3/users/b240f68acc734bd492843ad4ac35807c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:53.860088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b240f68acc734bd492843ad4ac35807c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:53.866390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:53.981228 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b240f68acc734bd492843ad4ac35807c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:53.991651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b240f68acc734bd492843ad4ac35807c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:53.992674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b240f68acc734bd492843ad4ac35807c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.008273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b240f68acc734bd492843ad4ac35807c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.009089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b240f68acc734bd492843ad4ac35807c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.028214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1059/2158] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:53 2022] DELETE /identity/v3/users/b240f68acc734bd492843ad4ac35807c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:54.216670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:54.218326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:54.238491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480e1a-2b08-4443-8b8f-f82f09ccf85b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:54.376626 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:54.379546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:54.379998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:54.380350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] PATH_INFO: `/v3/projects/5fd781c3511a40cc897d2968d178c1b6` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:54.383213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd781c3511a40cc897d2968d178c1b6)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:54.387422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:54.452638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:54.454232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:54.472463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd781c3511a40cc897d2968d178c1b6', 'request_id': 'req-db36c161-6de5-4775-ba35-d2232e4b71ba'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.478286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef5fa-fc8e-4ae6-a531-48488b91020d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:54.482113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd781c3511a40cc897d2968d178c1b6', 'request_id': 'req-db36c161-6de5-4775-ba35-d2232e4b71ba'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.501209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd781c3511a40cc897d2968d178c1b6', 'request_id': 'req-db36c161-6de5-4775-ba35-d2232e4b71ba'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.529988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] The token cache is being invalidate because project 5fd781c3511a40cc897d2968d178c1b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:54.530578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.533431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.541908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1100/2159] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:54 2022] DELETE /identity/v3/projects/5fd781c3511a40cc897d2968d178c1b6 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:54.542821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:54.544820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:54.585020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:54.585709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:54.585990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:54.586261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] PATH_INFO: `/v3/projects/01e9d5e4d9344c40ac5b44c6adf81943` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:54.587528 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e9d5e4d9344c40ac5b44c6adf81943)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:54.589929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:54.607166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:54.645635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:54.673253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36c161-6de5-4775-ba35-d2232e4b71ba admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:54.691878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:54.692844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:54.693127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:54.694464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] PATH_INFO: `/v3/projects/a0e7a5dae78a4c10926c6fb66458c71b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:54.695913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e7a5dae78a4c10926c6fb66458c71b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:54.698072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:54.715470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e9d5e4d9344c40ac5b44c6adf81943', 'request_id': 'req-957aad77-f235-4361-843a-2ae3618b3f60'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.729953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e9d5e4d9344c40ac5b44c6adf81943', 'request_id': 'req-957aad77-f235-4361-843a-2ae3618b3f60'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.743230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e9d5e4d9344c40ac5b44c6adf81943', 'request_id': 'req-957aad77-f235-4361-843a-2ae3618b3f60'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.766441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] The token cache is being invalidate because project 01e9d5e4d9344c40ac5b44c6adf81943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:54.766861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.767621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.773321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1060/2160] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:54 2022] DELETE /identity/v3/projects/01e9d5e4d9344c40ac5b44c6adf81943 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:54.773321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:54.773321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:54.803985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e7a5dae78a4c10926c6fb66458c71b', 'request_id': 'req-45152277-2d76-4fd5-9cca-0e8b49de8cac'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.815460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e7a5dae78a4c10926c6fb66458c71b', 'request_id': 'req-45152277-2d76-4fd5-9cca-0e8b49de8cac'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.843366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e7a5dae78a4c10926c6fb66458c71b', 'request_id': 'req-45152277-2d76-4fd5-9cca-0e8b49de8cac'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.866491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957aad77-f235-4361-843a-2ae3618b3f60 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:54.873424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] The token cache is being invalidate because project a0e7a5dae78a4c10926c6fb66458c71b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:54.873766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.874634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-45152277-2d76-4fd5-9cca-0e8b49de8cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:54.876489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1101/2161] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:54 2022] DELETE /identity/v3/projects/a0e7a5dae78a4c10926c6fb66458c71b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:54.886258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1554-5a96-4ee9-9a92-94bb11b091ea None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:54.886971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1554-5a96-4ee9-9a92-94bb11b091ea None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:54.887432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1554-5a96-4ee9-9a92-94bb11b091ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:54.922528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a5ad1554-5a96-4ee9-9a92-94bb11b091ea None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:54.923971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a5ad1554-5a96-4ee9-9a92-94bb11b091ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:54.964943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:54.965731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:54.966963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:54.967181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] PATH_INFO: `/v3/users/920f1b5b4f384838b730db2e7342dd58` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:54.983982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920f1b5b4f384838b730db2e7342dd58)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:54.994333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.044545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1102/2162] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:55.047199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd5306-09c3-4998-bdae-46491d9b3c07 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.047688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd5306-09c3-4998-bdae-46491d9b3c07 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.048105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd5306-09c3-4998-bdae-46491d9b3c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.083804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d1cd5306-09c3-4998-bdae-46491d9b3c07 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:55.085370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d1cd5306-09c3-4998-bdae-46491d9b3c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:55.090860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920f1b5b4f384838b730db2e7342dd58'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.093987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:55.102750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1103/2163] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:55.102750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d5492-1470-461e-8361-92c25a26631e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.102750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d5492-1470-461e-8361-92c25a26631e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.102750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d5492-1470-461e-8361-92c25a26631e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.114500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920f1b5b4f384838b730db2e7342dd58'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.115967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920f1b5b4f384838b730db2e7342dd58'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.135755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-459d5492-1470-461e-8361-92c25a26631e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:55.137536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-459d5492-1470-461e-8361-92c25a26631e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:55.150921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1104/2164] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:55.153123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c74bca-9f19-4bf7-affd-15faaf872d2e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.153600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c74bca-9f19-4bf7-affd-15faaf872d2e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.154014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c74bca-9f19-4bf7-affd-15faaf872d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.157043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920f1b5b4f384838b730db2e7342dd58'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.158209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920f1b5b4f384838b730db2e7342dd58'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.177768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1061/2165] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:54 2022] DELETE /identity/v3/users/920f1b5b4f384838b730db2e7342dd58 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:55.180017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.182319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:55.191011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-07c74bca-9f19-4bf7-affd-15faaf872d2e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:55.192280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-07c74bca-9f19-4bf7-affd-15faaf872d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:55.314364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:55.336174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1105/2166] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:55.336174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac765d-090a-452c-ac44-ea5ebcb724be None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.336174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac765d-090a-452c-ac44-ea5ebcb724be None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.336174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac765d-090a-452c-ac44-ea5ebcb724be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.379626 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-71ac765d-090a-452c-ac44-ea5ebcb724be None None] MFA Rules not processed for user `f37a9c5000c54dada98baebc43684446`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:55.381942 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-71ac765d-090a-452c-ac44-ea5ebcb724be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:55.416096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37159c-b2ca-4539-abd1-c005099e4f49 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.423009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:55.423775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.425589 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.426256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.427820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:55.428277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:55.430705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.458875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1062/2167] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:55.460015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.491007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566b06d-8ac8-4bbb-8f7b-907c013ff0db admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.506123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:55.506775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.507063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.507527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.509603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:55.510319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:55.512260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.531059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1063/2168] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:55.532026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.545013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e635a24-8b1b-4417-b83d-8221e7a7e850 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.551602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:55.552229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.552575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.553029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.553753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:55.556299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.561540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1106/2169] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:55.563064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac765d-090a-452c-ac44-ea5ebcb724be None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.587321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac765d-090a-452c-ac44-ea5ebcb724be None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.595130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929bbf2-8402-4096-84fe-1c04d1808d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:55.601441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929bbf2-8402-4096-84fe-1c04d1808d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.601812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929bbf2-8402-4096-84fe-1c04d1808d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.602096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929bbf2-8402-4096-84fe-1c04d1808d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.604446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929bbf2-8402-4096-84fe-1c04d1808d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:55.607375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929bbf2-8402-4096-84fe-1c04d1808d0c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.625670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1064/2170] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:55 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:55.626617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.629051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:55.656210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1107/2171] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:55 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:55.657211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929bbf2-8402-4096-84fe-1c04d1808d0c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.728407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9877cf9-1336-425e-be39-a61af8792aaf admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.741564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929bbf2-8402-4096-84fe-1c04d1808d0c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.749191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c4623-5845-49e4-89d9-f839ad5a3838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:55.750610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c4623-5845-49e4-89d9-f839ad5a3838 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.751088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c4623-5845-49e4-89d9-f839ad5a3838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.751580 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c4623-5845-49e4-89d9-f839ad5a3838 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.752249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:55.753063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c4623-5845-49e4-89d9-f839ad5a3838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:55.755868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c4623-5845-49e4-89d9-f839ad5a3838 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.794044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1108/2172] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:55 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:55.794700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c4623-5845-49e4-89d9-f839ad5a3838 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.814075 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c4623-5845-49e4-89d9-f839ad5a3838 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.820260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:55.821183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.823725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.824231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.825217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:55.827811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.838839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:55.840492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.840799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.841075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.843493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:55.854352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1109/2173] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:55 2022] POST /identity/v3/users => generated 286 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:55.855324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.857019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:55.872913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa950051-e6eb-4aa9-9af4-5fd49f45a855 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.880265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:55.880983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:55.881273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:55.881582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] PATH_INFO: `/v3/projects/b54deb60766e42fe93692996af85f9e4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:55.882847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b54deb60766e42fe93692996af85f9e4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:55.884766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.913570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=7,keystone:DELETE=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:55.926078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:55.927732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:55.938785 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1065/2174] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:55.939641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:55.947383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b54deb60766e42fe93692996af85f9e4', 'request_id': 'req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.962294 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d872d-4478-4272-8b78-f0bc29be320f service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:55.965857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b54deb60766e42fe93692996af85f9e4', 'request_id': 'req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.976557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b54deb60766e42fe93692996af85f9e4', 'request_id': 'req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.993791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] The token cache is being invalidate because project b54deb60766e42fe93692996af85f9e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:55.994125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:55.995316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:56.001781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1110/2175] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:55 2022] DELETE /identity/v3/projects/b54deb60766e42fe93692996af85f9e4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:56.002838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.005118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:56.043050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.043687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.043975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.044252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.044957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.046698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.060287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1066/2176] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:56.061288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.063751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:56.078692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77251d48-707d-4d6b-b089-0bd56d87dbbb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:56.089424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.090501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.090917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.091513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] PATH_INFO: `/v3/users/db042afc07424460b317e784df11505e` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.094472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=db042afc07424460b317e784df11505e)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.097468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.108402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe18b9f-4834-4c11-9588-a912fe0cc2a9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:56.115758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.116397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.116682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.117008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.117781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.120669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.138111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1111/2177] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:56.139410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.141480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:56.155999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332e24b-59ac-4af9-ad48-0bd160774e12 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:56.164047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f18ce-1dbf-47ca-af3c-6eabc17e7a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.166038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f18ce-1dbf-47ca-af3c-6eabc17e7a14 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.166512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f18ce-1dbf-47ca-af3c-6eabc17e7a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.166826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f18ce-1dbf-47ca-af3c-6eabc17e7a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.169751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f18ce-1dbf-47ca-af3c-6eabc17e7a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.172034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f18ce-1dbf-47ca-af3c-6eabc17e7a14 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.172837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db042afc07424460b317e784df11505e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:56.178292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:56.183215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db042afc07424460b317e784df11505e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:56.184856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db042afc07424460b317e784df11505e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:56.187310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1112/2178] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:56.187310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f18ce-1dbf-47ca-af3c-6eabc17e7a14 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.201956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f18ce-1dbf-47ca-af3c-6eabc17e7a14 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:56.206659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db042afc07424460b317e784df11505e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:56.207429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db042afc07424460b317e784df11505e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:56.207737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.208373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.208657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.209023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.209766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.211480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.228088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1067/2179] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:56 2022] DELETE /identity/v3/users/db042afc07424460b317e784df11505e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:56.228088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.233318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1113/2180] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:56.236461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:56.237747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.244936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:56.271336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a486ed-b8ad-42a7-897a-0c964c00c385 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:56.385577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873951b1-36a4-4faf-941a-898752093c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.386998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873951b1-36a4-4faf-941a-898752093c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.388133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190bc65-7c87-4e66-89c1-26a1e8312a7d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:56.388630 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873951b1-36a4-4faf-941a-898752093c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.389798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873951b1-36a4-4faf-941a-898752093c93 admin admin] PATH_INFO: `/v3/projects/4b7138988d684c458b51016c87629b56/users/1d0885d7146b47818e06125d931241ab/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.402313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b466c-e8cb-49fc-84a8-fd77e0bc0ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.403291 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b466c-e8cb-49fc-84a8-fd77e0bc0ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.403658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b466c-e8cb-49fc-84a8-fd77e0bc0ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.403937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b466c-e8cb-49fc-84a8-fd77e0bc0ed8 admin admin] PATH_INFO: `/v3/projects/db1d58a7f5ba40a89ebb256e531d8810/users/d86de6b2c6db4e9f9c68dd0fbbe3f5ad/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.411697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873951b1-36a4-4faf-941a-898752093c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7138988d684c458b51016c87629b56, user_id=1d0885d7146b47818e06125d931241ab, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.414896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873951b1-36a4-4faf-941a-898752093c93 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.429208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b466c-e8cb-49fc-84a8-fd77e0bc0ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1d58a7f5ba40a89ebb256e531d8810, user_id=d86de6b2c6db4e9f9c68dd0fbbe3f5ad, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.433472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b466c-e8cb-49fc-84a8-fd77e0bc0ed8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.459684 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1114/2181] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:56 2022] PUT /identity/v3/projects/4b7138988d684c458b51016c87629b56/users/1d0885d7146b47818e06125d931241ab/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:56.477491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1068/2182] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:56 2022] PUT /identity/v3/projects/db1d58a7f5ba40a89ebb256e531d8810/users/d86de6b2c6db4e9f9c68dd0fbbe3f5ad/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:56.477792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cac4-8eeb-47d0-9be0-2eb0e7992c50 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.477792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cac4-8eeb-47d0-9be0-2eb0e7992c50 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.477792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cac4-8eeb-47d0-9be0-2eb0e7992c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.481116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82010bba-dbbe-4466-8bfd-9ceae2b3e9f0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.481976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82010bba-dbbe-4466-8bfd-9ceae2b3e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.481976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82010bba-dbbe-4466-8bfd-9ceae2b3e9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.522159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-82010bba-dbbe-4466-8bfd-9ceae2b3e9f0 None None] MFA Rules not processed for user `d86de6b2c6db4e9f9c68dd0fbbe3f5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:56.523762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-82010bba-dbbe-4466-8bfd-9ceae2b3e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:56.546342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:56.563323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-26c7cac4-8eeb-47d0-9be0-2eb0e7992c50 None None] MFA Rules not processed for user `1d0885d7146b47818e06125d931241ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:56.565437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-26c7cac4-8eeb-47d0-9be0-2eb0e7992c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:56.665793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1069/2183] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:38:56.667050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82010bba-dbbe-4466-8bfd-9ceae2b3e9f0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.673671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1115/2184] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:56.679829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cac4-8eeb-47d0-9be0-2eb0e7992c50 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.753952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:56.788140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82010bba-dbbe-4466-8bfd-9ceae2b3e9f0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:56.794028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.794749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.795155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.795433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.796178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.796502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cac4-8eeb-47d0-9be0-2eb0e7992c50 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:56.799399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.812787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ef750-f625-4bf4-9196-b261cf1abcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:56.814212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ef750-f625-4bf4-9196-b261cf1abcdd admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:56.814508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ef750-f625-4bf4-9196-b261cf1abcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:56.814883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ef750-f625-4bf4-9196-b261cf1abcdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:56.816869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ef750-f625-4bf4-9196-b261cf1abcdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:56.820939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ef750-f625-4bf4-9196-b261cf1abcdd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:56.851922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1070/2185] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:56 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:56.855020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.859685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:56.898144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1116/2186] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:56 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:56.899609 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ef750-f625-4bf4-9196-b261cf1abcdd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:56.984117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e588f-0f96-4cd0-a76b-9652fae025df admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.005423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ef750-f625-4bf4-9196-b261cf1abcdd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.011691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759e424-b400-4060-b556-a3326494055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.012090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759e424-b400-4060-b556-a3326494055d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.012379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759e424-b400-4060-b556-a3326494055d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.012670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759e424-b400-4060-b556-a3326494055d admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.013421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759e424-b400-4060-b556-a3326494055d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.015359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759e424-b400-4060-b556-a3326494055d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.043761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1117/2187] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:56 2022] POST /identity/v3/users => generated 297 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:57.044689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759e424-b400-4060-b556-a3326494055d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.059773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759e424-b400-4060-b556-a3326494055d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.074070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.074845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.075282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.075605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] PATH_INFO: `/v3/projects/f2e176938463495bb87ecea601f3687b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.076062 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac393f-9c7f-42e8-9e65-ed40d80926f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.076830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac393f-9c7f-42e8-9e65-ed40d80926f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.076951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e176938463495bb87ecea601f3687b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.077325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac393f-9c7f-42e8-9e65-ed40d80926f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.077752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac393f-9c7f-42e8-9e65-ed40d80926f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.078559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac393f-9c7f-42e8-9e65-ed40d80926f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.081369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.082536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac393f-9c7f-42e8-9e65-ed40d80926f6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.095200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:57.119378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1118/2188] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:57 2022] POST /identity/v3/users => generated 293 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:57.119378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac393f-9c7f-42e8-9e65-ed40d80926f6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.132138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac393f-9c7f-42e8-9e65-ed40d80926f6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.156928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e176938463495bb87ecea601f3687b', 'request_id': 'req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:57.162571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdec630-765e-4f27-a332-20e7e9484468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.163507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdec630-765e-4f27-a332-20e7e9484468 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.168136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e176938463495bb87ecea601f3687b', 'request_id': 'req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:57.175321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdec630-765e-4f27-a332-20e7e9484468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.175321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdec630-765e-4f27-a332-20e7e9484468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.176358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdec630-765e-4f27-a332-20e7e9484468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.178360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdec630-765e-4f27-a332-20e7e9484468 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.180766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e176938463495bb87ecea601f3687b', 'request_id': 'req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:57.198351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] The token cache is being invalidate because project f2e176938463495bb87ecea601f3687b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:38:57.198903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:57.199929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:57.201800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1071/2189] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:56 2022] DELETE /identity/v3/projects/f2e176938463495bb87ecea601f3687b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:57.202747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.204218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:57.209251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1119/2190] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:57.210105 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f961df-df81-4017-b854-533770fe5471 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.210539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f961df-df81-4017-b854-533770fe5471 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.210950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f961df-df81-4017-b854-533770fe5471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.212627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:57.266464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e3f961df-df81-4017-b854-533770fe5471 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:57.267815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e3f961df-df81-4017-b854-533770fe5471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:57.308615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424ee1d-c6d3-46ea-a75e-8091624ea3c0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.314353 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.315034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.315379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.315379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.316536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.318217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.335546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1072/2191] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:57.336452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.339441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:57.363042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f55564-43a2-4138-adf0-4c0fee1539fe admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.368708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e73d3d-7315-4fd8-b10b-e3ac1f7b5005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.369495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73d3d-7315-4fd8-b10b-e3ac1f7b5005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.369787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73d3d-7315-4fd8-b10b-e3ac1f7b5005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.376317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73d3d-7315-4fd8-b10b-e3ac1f7b5005 admin admin] PATH_INFO: `/v3/projects/a6ee4cacd5064113b7705630554e6175/users/3279b2dff7674b68b44ce80d8669ca62/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.395320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1120/2192] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:57.396664 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f961df-df81-4017-b854-533770fe5471 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.412474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e73d3d-7315-4fd8-b10b-e3ac1f7b5005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ee4cacd5064113b7705630554e6175, user_id=3279b2dff7674b68b44ce80d8669ca62, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.417666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e73d3d-7315-4fd8-b10b-e3ac1f7b5005 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.422435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f961df-df81-4017-b854-533770fe5471 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.446367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f25e9b-e365-49de-919a-71867eae205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.450198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f25e9b-e365-49de-919a-71867eae205c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.450721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f25e9b-e365-49de-919a-71867eae205c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.451018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f25e9b-e365-49de-919a-71867eae205c admin admin] PATH_INFO: `/v3/projects/ad670b4b698a4809b09853c79ebe34f4/users/471490ce19e549b6ade5c2bd0a7b8d47/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.455237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1073/2193] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:57 2022] PUT /identity/v3/projects/a6ee4cacd5064113b7705630554e6175/users/3279b2dff7674b68b44ce80d8669ca62/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:57.460564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e73d3d-7315-4fd8-b10b-e3ac1f7b5005 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.493721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f25e9b-e365-49de-919a-71867eae205c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad670b4b698a4809b09853c79ebe34f4, user_id=471490ce19e549b6ade5c2bd0a7b8d47, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.497365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f25e9b-e365-49de-919a-71867eae205c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.525328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1121/2194] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:57 2022] PUT /identity/v3/projects/ad670b4b698a4809b09853c79ebe34f4/users/471490ce19e549b6ade5c2bd0a7b8d47/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:57.526809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d54284-103d-45b1-86eb-29353c6768ed None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.527101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d54284-103d-45b1-86eb-29353c6768ed None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.527379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d54284-103d-45b1-86eb-29353c6768ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.556887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e8d54284-103d-45b1-86eb-29353c6768ed None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:57.558868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e8d54284-103d-45b1-86eb-29353c6768ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:57.596085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e73d3d-7315-4fd8-b10b-e3ac1f7b5005 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.635412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1122/2195] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:57.636554 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d54284-103d-45b1-86eb-29353c6768ed None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.648991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d54284-103d-45b1-86eb-29353c6768ed None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.656441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41df92-547a-464d-9a12-81c5ce7dd497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.657172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41df92-547a-464d-9a12-81c5ce7dd497 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.657819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41df92-547a-464d-9a12-81c5ce7dd497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.658103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41df92-547a-464d-9a12-81c5ce7dd497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.659791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41df92-547a-464d-9a12-81c5ce7dd497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.660843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41df92-547a-464d-9a12-81c5ce7dd497 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.678723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1123/2196] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:57.679651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41df92-547a-464d-9a12-81c5ce7dd497 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.693424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d897d-1fe7-48fb-8629-9137931e46c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.693730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41df92-547a-464d-9a12-81c5ce7dd497 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.694498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d897d-1fe7-48fb-8629-9137931e46c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.694878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d897d-1fe7-48fb-8629-9137931e46c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.695360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d897d-1fe7-48fb-8629-9137931e46c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.696349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d897d-1fe7-48fb-8629-9137931e46c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.698887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d897d-1fe7-48fb-8629-9137931e46c1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.699809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.700614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.700989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.701347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.702500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:38:57.702924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.705154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.724208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1124/2197] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:38:57.727857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.731704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1074/2198] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:57.732781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d897d-1fe7-48fb-8629-9137931e46c1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.738470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202c3d-8927-4797-bd94-64b5f2f77416 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.743954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f41eb2-543f-4889-9b1b-f953445b102b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.744638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f41eb2-543f-4889-9b1b-f953445b102b admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.744917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f41eb2-543f-4889-9b1b-f953445b102b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.745188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f41eb2-543f-4889-9b1b-f953445b102b admin admin] PATH_INFO: `/v3/projects/ad670b4b698a4809b09853c79ebe34f4/users/471490ce19e549b6ade5c2bd0a7b8d47/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.745342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d897d-1fe7-48fb-8629-9137931e46c1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.747844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f41eb2-543f-4889-9b1b-f953445b102b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad670b4b698a4809b09853c79ebe34f4, user_id=471490ce19e549b6ade5c2bd0a7b8d47, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.751299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f41eb2-543f-4889-9b1b-f953445b102b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.762851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39f64a-dba5-424c-98af-9f99ef3f0b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.763399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39f64a-dba5-424c-98af-9f99ef3f0b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.763693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39f64a-dba5-424c-98af-9f99ef3f0b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.763971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39f64a-dba5-424c-98af-9f99ef3f0b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.764649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39f64a-dba5-424c-98af-9f99ef3f0b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.765972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1125/2199] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:57 2022] PUT /identity/v3/projects/ad670b4b698a4809b09853c79ebe34f4/users/471490ce19e549b6ade5c2bd0a7b8d47/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:57.766957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f41eb2-543f-4889-9b1b-f953445b102b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.770690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39f64a-dba5-424c-98af-9f99ef3f0b70 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.810329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:57.812925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:57.828937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1075/2200] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:38:57 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:57.830854 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64282ca-043c-43c0-9197-877a820d795f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.830996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64282ca-043c-43c0-9197-877a820d795f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.831669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64282ca-043c-43c0-9197-877a820d795f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.865030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f41eb2-543f-4889-9b1b-f953445b102b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:57.866515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b64282ca-043c-43c0-9197-877a820d795f None None] MFA Rules not processed for user `471490ce19e549b6ade5c2bd0a7b8d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:57.867861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b64282ca-043c-43c0-9197-877a820d795f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:57.930253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8d0ef-02c5-4895-bd56-9734d89aacbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:57.930952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8d0ef-02c5-4895-bd56-9734d89aacbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.931444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8d0ef-02c5-4895-bd56-9734d89aacbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.931581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8d0ef-02c5-4895-bd56-9734d89aacbb admin admin] PATH_INFO: `/v3/projects/a6ee4cacd5064113b7705630554e6175/users/3279b2dff7674b68b44ce80d8669ca62/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:57.934903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8d0ef-02c5-4895-bd56-9734d89aacbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ee4cacd5064113b7705630554e6175, user_id=3279b2dff7674b68b44ce80d8669ca62, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:57.936381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8d0ef-02c5-4895-bd56-9734d89aacbb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:57.948896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1126/2201] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:57 2022] PUT /identity/v3/projects/a6ee4cacd5064113b7705630554e6175/users/3279b2dff7674b68b44ce80d8669ca62/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:57.950424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8d0ef-02c5-4895-bd56-9734d89aacbb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:57.953486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1076/2202] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:57.968541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11709d-35d7-489e-b340-aa36e8e39e2b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:57.969053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11709d-35d7-489e-b340-aa36e8e39e2b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:57.969537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11709d-35d7-489e-b340-aa36e8e39e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.005457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-de11709d-35d7-489e-b340-aa36e8e39e2b None None] MFA Rules not processed for user `3279b2dff7674b68b44ce80d8669ca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:58.007206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-de11709d-35d7-489e-b340-aa36e8e39e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:58.046883 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8d0ef-02c5-4895-bd56-9734d89aacbb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.055309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.055950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.056237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.056517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.057440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.059278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:58.096016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1127/2203] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:38:57 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:38:58.098523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:58.100676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:58.109514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1077/2204] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:58.110477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11709d-35d7-489e-b340-aa36e8e39e2b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:58.126620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11709d-35d7-489e-b340-aa36e8e39e2b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.136472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224958b-47d1-41a4-ad36-4ca2e26421d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.137184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224958b-47d1-41a4-ad36-4ca2e26421d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.137634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224958b-47d1-41a4-ad36-4ca2e26421d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.138556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224958b-47d1-41a4-ad36-4ca2e26421d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.139397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224958b-47d1-41a4-ad36-4ca2e26421d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.142015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224958b-47d1-41a4-ad36-4ca2e26421d1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:58.158170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1078/2205] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:58.163024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224958b-47d1-41a4-ad36-4ca2e26421d1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:58.173611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224958b-47d1-41a4-ad36-4ca2e26421d1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.190942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.191400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.191683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.191959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.192658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.194768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:58.202229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b1323-23b4-4125-8837-7201d61b376f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.206920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb1202-e5d5-4672-bdaa-7f44c7044664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.207349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1079/2206] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:38:58.208188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:58.209919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:58.210774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb1202-e5d5-4672-bdaa-7f44c7044664 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.211108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb1202-e5d5-4672-bdaa-7f44c7044664 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.211337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb1202-e5d5-4672-bdaa-7f44c7044664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.220988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:58.223526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dce8ad-ab76-4626-b561-6f4a7b3e7ad0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.227534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173dc62-308f-466d-ad41-9c5d301c7ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.228158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173dc62-308f-466d-ad41-9c5d301c7ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.228486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173dc62-308f-466d-ad41-9c5d301c7ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.228804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173dc62-308f-466d-ad41-9c5d301c7ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.239428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173dc62-308f-466d-ad41-9c5d301c7ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.240859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173dc62-308f-466d-ad41-9c5d301c7ef6 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:58.260298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1080/2207] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:58.262563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173dc62-308f-466d-ad41-9c5d301c7ef6 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:58.277232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173dc62-308f-466d-ad41-9c5d301c7ef6 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.292373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.293062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.293377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.293652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] PATH_INFO: `/v3/projects/57601f159e244aa68924602c2c5dc385/users/ffc6812703254c51a4200abf3dee2d95/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.305708 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb1202-e5d5-4672-bdaa-7f44c7044664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.307347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb1202-e5d5-4672-bdaa-7f44c7044664 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:58.309946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] RBAC: Authorizing `identity:create_grant(project_id=57601f159e244aa68924602c2c5dc385, user_id=ffc6812703254c51a4200abf3dee2d95, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.312550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:58.318368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1128/2208] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:58.327347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1081/2209] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:38:58 2022] PUT /identity/v3/projects/57601f159e244aa68924602c2c5dc385/users/ffc6812703254c51a4200abf3dee2d95/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:58.332334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b759-da58-4c25-aa6d-ac627406e16f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.332633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b759-da58-4c25-aa6d-ac627406e16f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.332910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b759-da58-4c25-aa6d-ac627406e16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.353107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6b70b759-da58-4c25-aa6d-ac627406e16f None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:58.354443 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6b70b759-da58-4c25-aa6d-ac627406e16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:58.445238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1129/2210] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:58.456148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:58.458904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:58.470863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:58.482220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70b759-da58-4c25-aa6d-ac627406e16f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:58.548108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84318bdd-4817-42da-b6d8-b93f6c0efeac admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.556054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70b759-da58-4c25-aa6d-ac627406e16f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.559688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaa918-9936-4e9a-9722-6bb09132cd03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.560334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa918-9936-4e9a-9722-6bb09132cd03 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.560622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa918-9936-4e9a-9722-6bb09132cd03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.560897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa918-9936-4e9a-9722-6bb09132cd03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.561115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c61dda-ce65-4eb0-97e3-2c44c246650c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.562024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c61dda-ce65-4eb0-97e3-2c44c246650c service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.562540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c61dda-ce65-4eb0-97e3-2c44c246650c service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.562822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c61dda-ce65-4eb0-97e3-2c44c246650c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.564173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-14eaa918-9936-4e9a-9722-6bb09132cd03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:58.574553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c61dda-ce65-4eb0-97e3-2c44c246650c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.576131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c61dda-ce65-4eb0-97e3-2c44c246650c service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:58.586494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1082/2211] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:38:58.643763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eaa918-9936-4e9a-9722-6bb09132cd03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.645603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eaa918-9936-4e9a-9722-6bb09132cd03 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:58.651423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:58.660003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1130/2212] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:58.906074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c61dda-ce65-4eb0-97e3-2c44c246650c service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:58.930910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c61dda-ce65-4eb0-97e3-2c44c246650c service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:58.940422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:58.941550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:58.942039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:58.942527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] PATH_INFO: `/v3/users/1d0885d7146b47818e06125d931241ab` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:58.944737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0885d7146b47818e06125d931241ab)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:58.948283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:59.056567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0885d7146b47818e06125d931241ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:59.081485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0885d7146b47818e06125d931241ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:59.082556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0885d7146b47818e06125d931241ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:59.142349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0885d7146b47818e06125d931241ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:59.144558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0885d7146b47818e06125d931241ab'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:38:59.172121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1083/2213] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:38:58 2022] DELETE /identity/v3/users/1d0885d7146b47818e06125d931241ab => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:38:59.176634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2af46-abae-4feb-a47d-b79867c0fbec None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:59.177107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2af46-abae-4feb-a47d-b79867c0fbec None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:59.177345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2af46-abae-4feb-a47d-b79867c0fbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:59.203322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7db2af46-abae-4feb-a47d-b79867c0fbec None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:59.204581 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7db2af46-abae-4feb-a47d-b79867c0fbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:59.299655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1131/2214] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:59.311285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:59.312774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:59.434465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c6b1a-02f6-4daa-bb22-ae62fe155c18 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:59.439193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:59.440129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:59.440970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:59.441621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:59.464874 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:59.466375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:59.479897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1084/2215] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:38:59.646221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0f7bb-a3e5-412a-928c-dd513a9a16ea None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:59.647226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0f7bb-a3e5-412a-928c-dd513a9a16ea None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:59.647622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0f7bb-a3e5-412a-928c-dd513a9a16ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:59.686830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-40f0f7bb-a3e5-412a-928c-dd513a9a16ea None None] MFA Rules not processed for user `ffc6812703254c51a4200abf3dee2d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:38:59.687970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-40f0f7bb-a3e5-412a-928c-dd513a9a16ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:59.795654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1132/2216] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:38:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:38:59.804354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:59.806218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:59.853638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:38:59.899144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de1e37-078e-4db2-9791-344443bc1c62 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:59.903378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd03a9-43a8-476b-bbbc-caf3f0887034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:59.905823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd03a9-43a8-476b-bbbc-caf3f0887034 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:59.906435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd03a9-43a8-476b-bbbc-caf3f0887034 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:59.907415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd03a9-43a8-476b-bbbc-caf3f0887034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:59.913475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0f7bb-a3e5-412a-928c-dd513a9a16ea None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:38:59.914990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-40f0f7bb-a3e5-412a-928c-dd513a9a16ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:38:59.927243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd03a9-43a8-476b-bbbc-caf3f0887034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:59.928836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd03a9-43a8-476b-bbbc-caf3f0887034 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:59.935748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0f7bb-a3e5-412a-928c-dd513a9a16ea None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:38:59.944248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:38:59.945156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:38:59.945614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:38:59.946066 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] PATH_INFO: `/v3/projects/4b7138988d684c458b51016c87629b56` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:38:59.947658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7138988d684c458b51016c87629b56)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:38:59.950058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:38:59.962575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1085/2217] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:38:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:39:00.015174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7138988d684c458b51016c87629b56', 'request_id': 'req-732f6564-e74d-45c5-a2c2-5b980a2d1da3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.022542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7138988d684c458b51016c87629b56', 'request_id': 'req-732f6564-e74d-45c5-a2c2-5b980a2d1da3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.031186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7138988d684c458b51016c87629b56', 'request_id': 'req-732f6564-e74d-45c5-a2c2-5b980a2d1da3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.045592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] The token cache is being invalidate because project 4b7138988d684c458b51016c87629b56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:00.045929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.046707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.049015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1133/2218] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:38:59 2022] DELETE /identity/v3/projects/4b7138988d684c458b51016c87629b56 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:00.053188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd03a9-43a8-476b-bbbc-caf3f0887034 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:00.067261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd03a9-43a8-476b-bbbc-caf3f0887034 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:00.155232 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:00.155891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:00.156165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:00.156442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] PATH_INFO: `/v3/users/471490ce19e549b6ade5c2bd0a7b8d47` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:00.157939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=471490ce19e549b6ade5c2bd0a7b8d47)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:00.159912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:00.221959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471490ce19e549b6ade5c2bd0a7b8d47'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.233773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471490ce19e549b6ade5c2bd0a7b8d47'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.234502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471490ce19e549b6ade5c2bd0a7b8d47'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.264240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471490ce19e549b6ade5c2bd0a7b8d47'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.265927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f658b026-f7f3-4ecb-9619-866829f7c621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471490ce19e549b6ade5c2bd0a7b8d47'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:00.286011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1086/2219] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:00 2022] DELETE /identity/v3/users/471490ce19e549b6ade5c2bd0a7b8d47 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:39:00.877130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:00.880091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:00.895210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f6564-e74d-45c5-a2c2-5b980a2d1da3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:00.916769 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:00.982387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:00.983316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:00.983806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:00.984300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] PATH_INFO: `/v3/projects/ad670b4b698a4809b09853c79ebe34f4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:00.985786 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad670b4b698a4809b09853c79ebe34f4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:00.988035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:01.050129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad670b4b698a4809b09853c79ebe34f4', 'request_id': 'req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:01.055899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad670b4b698a4809b09853c79ebe34f4', 'request_id': 'req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:01.064017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad670b4b698a4809b09853c79ebe34f4', 'request_id': 'req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:01.076470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] The token cache is being invalidate because project ad670b4b698a4809b09853c79ebe34f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:01.076762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:01.077458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2d98bb6d-227a-4427-81f4-0ac00dfc38b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:01.078983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1134/2220] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:00 2022] DELETE /identity/v3/projects/ad670b4b698a4809b09853c79ebe34f4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:01.085543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c040f-6e9c-47d5-8618-90cf2c7f1a93 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.086135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c040f-6e9c-47d5-8618-90cf2c7f1a93 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.086135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c040f-6e9c-47d5-8618-90cf2c7f1a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.108193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e89c040f-6e9c-47d5-8618-90cf2c7f1a93 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:01.109481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e89c040f-6e9c-47d5-8618-90cf2c7f1a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:01.163847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:01.187768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1087/2221] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:01.196572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afda5a-c538-4fb1-8aeb-182c3660e6e9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.196922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afda5a-c538-4fb1-8aeb-182c3660e6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.197190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afda5a-c538-4fb1-8aeb-182c3660e6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.218380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c2afda5a-c538-4fb1-8aeb-182c3660e6e9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:01.219599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c2afda5a-c538-4fb1-8aeb-182c3660e6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:01.228428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1135/2222] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:01.232726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c040f-6e9c-47d5-8618-90cf2c7f1a93 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:01.243230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c040f-6e9c-47d5-8618-90cf2c7f1a93 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:01.248453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:01.249613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.249613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.249790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.250569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:39:01.251091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:01.253630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:01.266636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1088/2223] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:39:01.271748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afda5a-c538-4fb1-8aeb-182c3660e6e9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:01.281371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afda5a-c538-4fb1-8aeb-182c3660e6e9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:01.286847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92230-b3b9-4ce7-8088-c423a319c71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:01.287568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92230-b3b9-4ce7-8088-c423a319c71d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.287911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92230-b3b9-4ce7-8088-c423a319c71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.288251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92230-b3b9-4ce7-8088-c423a319c71d admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.288991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e92230-b3b9-4ce7-8088-c423a319c71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:01.291066 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e92230-b3b9-4ce7-8088-c423a319c71d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:01.327510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1136/2224] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:01 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:01.333705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:01.346960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238ec07-e065-40c5-bf40-cec682d662d5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:01.412457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197afcd-949d-4249-9ba4-0494433699ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:01.412975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197afcd-949d-4249-9ba4-0494433699ef admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.413264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197afcd-949d-4249-9ba4-0494433699ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.413618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197afcd-949d-4249-9ba4-0494433699ef admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.414338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197afcd-949d-4249-9ba4-0494433699ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:01.416692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197afcd-949d-4249-9ba4-0494433699ef admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:01.460941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1089/2225] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:01 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:01.473322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92230-b3b9-4ce7-8088-c423a319c71d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:01.482658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92230-b3b9-4ce7-8088-c423a319c71d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:01.486028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8beb3-3cfd-47ad-a9e2-813d5459f0a6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.486319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a81cd8-379c-4f8f-b869-a061e762aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:01.487981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a81cd8-379c-4f8f-b869-a061e762aaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.487981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a81cd8-379c-4f8f-b869-a061e762aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.488429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8beb3-3cfd-47ad-a9e2-813d5459f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.488537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a81cd8-379c-4f8f-b869-a061e762aaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.489009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8beb3-3cfd-47ad-a9e2-813d5459f0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.489339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a81cd8-379c-4f8f-b869-a061e762aaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:01.491018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a81cd8-379c-4f8f-b869-a061e762aaa1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:01.508158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1137/2226] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:01.509312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a81cd8-379c-4f8f-b869-a061e762aaa1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:01.516101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-19b8beb3-3cfd-47ad-a9e2-813d5459f0a6 None None] MFA Rules not processed for user `ffc6812703254c51a4200abf3dee2d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:01.517554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-19b8beb3-3cfd-47ad-a9e2-813d5459f0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:01.526509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a81cd8-379c-4f8f-b869-a061e762aaa1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:01.539400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1e48f-8803-4d49-9f98-e8b86de9e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:01.540035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e48f-8803-4d49-9f98-e8b86de9e5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.540316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e48f-8803-4d49-9f98-e8b86de9e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.540593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e48f-8803-4d49-9f98-e8b86de9e5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.541314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1e48f-8803-4d49-9f98-e8b86de9e5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:01.542958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1e48f-8803-4d49-9f98-e8b86de9e5dd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:01.559352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1138/2227] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:01.560771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1e48f-8803-4d49-9f98-e8b86de9e5dd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:01.595302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1e48f-8803-4d49-9f98-e8b86de9e5dd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:01.603421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1090/2228] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:01.608069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd794369-0687-466f-9670-f9f9820dfa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:01.608809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd794369-0687-466f-9670-f9f9820dfa42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.609211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd794369-0687-466f-9670-f9f9820dfa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.609628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd794369-0687-466f-9670-f9f9820dfa42 admin admin] PATH_INFO: `/v3/projects/1ce064f4048447e89f4ae0ccb1d916a5/users/61ae40b2616545bea6f224ac2559731c/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.631256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd794369-0687-466f-9670-f9f9820dfa42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce064f4048447e89f4ae0ccb1d916a5, user_id=61ae40b2616545bea6f224ac2559731c, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:01.634017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd794369-0687-466f-9670-f9f9820dfa42 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:01.660014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1139/2229] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:01 2022] PUT /identity/v3/projects/1ce064f4048447e89f4ae0ccb1d916a5/users/61ae40b2616545bea6f224ac2559731c/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:01.674028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46066-df84-4731-b244-953129283a88 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.674945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46066-df84-4731-b244-953129283a88 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.675291 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46066-df84-4731-b244-953129283a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.731345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-47d46066-df84-4731-b244-953129283a88 None None] MFA Rules not processed for user `61ae40b2616545bea6f224ac2559731c`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:01.732734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-47d46066-df84-4731-b244-953129283a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:01.819987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:01.827275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1091/2230] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:01.835210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd794369-0687-466f-9670-f9f9820dfa42 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:01.849124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d46066-df84-4731-b244-953129283a88 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:01.851261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-47d46066-df84-4731-b244-953129283a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:01.911772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:01.974361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd794369-0687-466f-9670-f9f9820dfa42 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:01.984293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7aaaea-fba7-44f1-830f-7c55118a8c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:01.985396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aaaea-fba7-44f1-830f-7c55118a8c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:01.985863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aaaea-fba7-44f1-830f-7c55118a8c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:01.986425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aaaea-fba7-44f1-830f-7c55118a8c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:01.987523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7aaaea-fba7-44f1-830f-7c55118a8c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:01.989925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7aaaea-fba7-44f1-830f-7c55118a8c49 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:02.020189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d46066-df84-4731-b244-953129283a88 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:02.025500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:02.026268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.026619 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.027028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] PATH_INFO: `/v3/users/d86de6b2c6db4e9f9c68dd0fbbe3f5ad` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.028269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86de6b2c6db4e9f9c68dd0fbbe3f5ad)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:02.030504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:02.055948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1140/2231] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:01 2022] POST /identity/v3/projects => generated 387 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:02.061172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7aaaea-fba7-44f1-830f-7c55118a8c49 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:02.116882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86de6b2c6db4e9f9c68dd0fbbe3f5ad'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:02.135761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86de6b2c6db4e9f9c68dd0fbbe3f5ad'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:02.137517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86de6b2c6db4e9f9c68dd0fbbe3f5ad'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:02.161105 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86de6b2c6db4e9f9c68dd0fbbe3f5ad'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:02.162087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a0a1185d-3175-493f-9cbe-f2999e51ae7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86de6b2c6db4e9f9c68dd0fbbe3f5ad'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:02.195274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1092/2232] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:01 2022] DELETE /identity/v3/users/d86de6b2c6db4e9f9c68dd0fbbe3f5ad => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:39:02.197501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7aaaea-fba7-44f1-830f-7c55118a8c49 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:02.199770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addf47-ca5b-4c63-bab9-e05c7dca4e3c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.200125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addf47-ca5b-4c63-bab9-e05c7dca4e3c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.200407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addf47-ca5b-4c63-bab9-e05c7dca4e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.224198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-32addf47-ca5b-4c63-bab9-e05c7dca4e3c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:02.225595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-32addf47-ca5b-4c63-bab9-e05c7dca4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:02.275867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:02.276566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.277094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.277452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.278203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:02.281232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:02.315781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1141/2233] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:02 2022] POST /identity/v3/users => generated 300 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:39:02.317434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:02.320018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:02.324631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1093/2234] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:02.335815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4924-a118-4e49-aa6b-357ea12475b7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:02.336345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32addf47-ca5b-4c63-bab9-e05c7dca4e3c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:02.338585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-32addf47-ca5b-4c63-bab9-e05c7dca4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:02.341025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a642-369a-4293-85c1-58148bd0a5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:02.341861 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a642-369a-4293-85c1-58148bd0a5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.342181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a642-369a-4293-85c1-58148bd0a5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.342425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a642-369a-4293-85c1-58148bd0a5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.343273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4a642-369a-4293-85c1-58148bd0a5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:02.344782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4a642-369a-4293-85c1-58148bd0a5d1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:02.358945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1142/2235] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:39:02.371372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a642-369a-4293-85c1-58148bd0a5d1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:02.373775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:02.392954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a642-369a-4293-85c1-58148bd0a5d1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:02.401407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407a402-1371-4060-a33d-66d48d57d013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:02.402396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407a402-1371-4060-a33d-66d48d57d013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.403101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407a402-1371-4060-a33d-66d48d57d013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.404347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407a402-1371-4060-a33d-66d48d57d013 admin admin] PATH_INFO: `/v3/projects/c1870121634d49daa0e365229fa42138/users/437a35c22aa94d0eb8b1e487c6b47d6d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.424816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407a402-1371-4060-a33d-66d48d57d013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1870121634d49daa0e365229fa42138, user_id=437a35c22aa94d0eb8b1e487c6b47d6d, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:02.427899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407a402-1371-4060-a33d-66d48d57d013 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:02.444678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32addf47-ca5b-4c63-bab9-e05c7dca4e3c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:02.451327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1143/2236] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:02 2022] PUT /identity/v3/projects/c1870121634d49daa0e365229fa42138/users/437a35c22aa94d0eb8b1e487c6b47d6d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:02.456024 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407a402-1371-4060-a33d-66d48d57d013 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:02.483021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:02.531607 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a69ac3-e6dd-4de6-ab7f-c0170c04a048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:02.532650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a69ac3-e6dd-4de6-ab7f-c0170c04a048 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.532950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a69ac3-e6dd-4de6-ab7f-c0170c04a048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.533428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a69ac3-e6dd-4de6-ab7f-c0170c04a048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.567588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407a402-1371-4060-a33d-66d48d57d013 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:02.582931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32de21e-95ed-4974-a6ef-eb79cd473e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:02.583568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32de21e-95ed-4974-a6ef-eb79cd473e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.583899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32de21e-95ed-4974-a6ef-eb79cd473e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.584204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32de21e-95ed-4974-a6ef-eb79cd473e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.585602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32de21e-95ed-4974-a6ef-eb79cd473e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:02.587058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32de21e-95ed-4974-a6ef-eb79cd473e75 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:02.620197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1144/2237] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:02.621496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32de21e-95ed-4974-a6ef-eb79cd473e75 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:02.647192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32de21e-95ed-4974-a6ef-eb79cd473e75 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:02.666923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:02.667712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.668005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.668284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] PATH_INFO: `/v3/projects/c1870121634d49daa0e365229fa42138/users/437a35c22aa94d0eb8b1e487c6b47d6d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.670517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1870121634d49daa0e365229fa42138, user_id=437a35c22aa94d0eb8b1e487c6b47d6d, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:02.676873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:02.688629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a69ac3-e6dd-4de6-ab7f-c0170c04a048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:02.690876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a69ac3-e6dd-4de6-ab7f-c0170c04a048 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:02.702254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1094/2238] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:39:02.717610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1145/2239] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:02 2022] PUT /identity/v3/projects/c1870121634d49daa0e365229fa42138/users/437a35c22aa94d0eb8b1e487c6b47d6d/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:02.722072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13679a5-4d2c-4c05-a4a1-b583157dc7a8 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:02.722368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13679a5-4d2c-4c05-a4a1-b583157dc7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:02.722839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13679a5-4d2c-4c05-a4a1-b583157dc7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:02.779250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f13679a5-4d2c-4c05-a4a1-b583157dc7a8 None None] MFA Rules not processed for user `437a35c22aa94d0eb8b1e487c6b47d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:02.780645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f13679a5-4d2c-4c05-a4a1-b583157dc7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:02.890405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1095/2240] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:02.918810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:02.932047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:03.029449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2540-8832-4e03-86d4-233df71c3289 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:03.033208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e737114-4664-453c-95ac-3a3dad862666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:03.034035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e737114-4664-453c-95ac-3a3dad862666 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.034530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e737114-4664-453c-95ac-3a3dad862666 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.034812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e737114-4664-453c-95ac-3a3dad862666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.045156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e737114-4664-453c-95ac-3a3dad862666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:03.047668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e737114-4664-453c-95ac-3a3dad862666 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:03.057983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1146/2241] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:03.072176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13679a5-4d2c-4c05-a4a1-b583157dc7a8 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:03.074180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f13679a5-4d2c-4c05-a4a1-b583157dc7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:03.102168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:03.139063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e737114-4664-453c-95ac-3a3dad862666 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:03.210782 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13679a5-4d2c-4c05-a4a1-b583157dc7a8 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:03.219426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:03.220287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.220472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.220734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] PATH_INFO: `/v3/projects/db1d58a7f5ba40a89ebb256e531d8810` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.222872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1d58a7f5ba40a89ebb256e531d8810)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:03.224942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:03.247673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e737114-4664-453c-95ac-3a3dad862666 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:03.256470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:03.257892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.258181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.258641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] PATH_INFO: `/v3/users/61ae40b2616545bea6f224ac2559731c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.260315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ae40b2616545bea6f224ac2559731c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:03.262657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:03.285962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1d58a7f5ba40a89ebb256e531d8810', 'request_id': 'req-0a8986e1-8e96-4841-82f0-69bb92da9c3c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.296189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1d58a7f5ba40a89ebb256e531d8810', 'request_id': 'req-0a8986e1-8e96-4841-82f0-69bb92da9c3c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.306525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1d58a7f5ba40a89ebb256e531d8810', 'request_id': 'req-0a8986e1-8e96-4841-82f0-69bb92da9c3c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.324952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:03.329052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] The token cache is being invalidate because project db1d58a7f5ba40a89ebb256e531d8810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:03.329536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ae40b2616545bea6f224ac2559731c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.329804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.330639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.333346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1096/2242] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:03 2022] DELETE /identity/v3/projects/db1d58a7f5ba40a89ebb256e531d8810 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:03.338848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:03.341208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:03.344163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ae40b2616545bea6f224ac2559731c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.362662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ae40b2616545bea6f224ac2559731c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.364034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ae40b2616545bea6f224ac2559731c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.382644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ae40b2616545bea6f224ac2559731c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.391778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1147/2243] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:03 2022] DELETE /identity/v3/users/61ae40b2616545bea6f224ac2559731c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:03.393015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:03.394612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:03.420458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8986e1-8e96-4841-82f0-69bb92da9c3c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:03.467432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:03.478902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9202ec4-e5b7-4dfe-9aca-14923f1bd0cc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:03.482468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:03.483149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.483435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.483612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:03.483725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] PATH_INFO: `/v3/users/dbd454d2a4b542689c0437eb4dbefc02` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.484257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.484539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.484815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] PATH_INFO: `/v3/users/3279b2dff7674b68b44ce80d8669ca62` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.485121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd454d2a4b542689c0437eb4dbefc02)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:03.486045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3279b2dff7674b68b44ce80d8669ca62)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:03.488012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:03.489051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:03.539241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd454d2a4b542689c0437eb4dbefc02'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.543708 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3279b2dff7674b68b44ce80d8669ca62'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.549026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd454d2a4b542689c0437eb4dbefc02'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.550790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd454d2a4b542689c0437eb4dbefc02'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.552102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3279b2dff7674b68b44ce80d8669ca62'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.561375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3279b2dff7674b68b44ce80d8669ca62'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.562312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3279b2dff7674b68b44ce80d8669ca62'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.574397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd454d2a4b542689c0437eb4dbefc02'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.575307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0ccd2d8-e1dd-4ebf-b821-9832fb57c418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd454d2a4b542689c0437eb4dbefc02'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.593800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1097/2244] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:03 2022] DELETE /identity/v3/users/dbd454d2a4b542689c0437eb4dbefc02 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:03.597641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f8145-4840-498c-96fa-99e2844fbdae None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.597641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f8145-4840-498c-96fa-99e2844fbdae None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.597641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f8145-4840-498c-96fa-99e2844fbdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.600509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d0c6d742-5def-49c3-971f-919d71533a01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3279b2dff7674b68b44ce80d8669ca62'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:03.608796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1148/2245] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:03 2022] DELETE /identity/v3/users/3279b2dff7674b68b44ce80d8669ca62 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:03.610444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aa8df-504b-478a-b564-200334ffd560 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.610740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aa8df-504b-478a-b564-200334ffd560 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.611022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aa8df-504b-478a-b564-200334ffd560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.637082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-618f8145-4840-498c-96fa-99e2844fbdae None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:03.637082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-618f8145-4840-498c-96fa-99e2844fbdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:03.637457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7c8aa8df-504b-478a-b564-200334ffd560 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:03.637457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7c8aa8df-504b-478a-b564-200334ffd560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:03.680953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:03.709223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1098/2246] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:03.719189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f8145-4840-498c-96fa-99e2844fbdae None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:03.721476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-618f8145-4840-498c-96fa-99e2844fbdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:03.723500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1149/2247] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:03.734151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aa8df-504b-478a-b564-200334ffd560 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:03.735418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7c8aa8df-504b-478a-b564-200334ffd560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:03.826795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f8145-4840-498c-96fa-99e2844fbdae None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:03.834752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:03.835644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.836110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.836567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.853140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:03.855185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:03.868546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1099/2248] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:03.868670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aa8df-504b-478a-b564-200334ffd560 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:03.873563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6a598-6e90-4420-8e12-702569ea4458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:03.874208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a598-6e90-4420-8e12-702569ea4458 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:03.874788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a598-6e90-4420-8e12-702569ea4458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:03.874788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a598-6e90-4420-8e12-702569ea4458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:03.885875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6a598-6e90-4420-8e12-702569ea4458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:03.887314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6a598-6e90-4420-8e12-702569ea4458 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:03.898445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1150/2249] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:39:04.070106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:04.071599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.085498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ce55f-9ed5-4af1-854c-2b94288909c7 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:04.090199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:04.090838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.091115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.091388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] PATH_INFO: `/v3/projects/a6ee4cacd5064113b7705630554e6175` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.092658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ee4cacd5064113b7705630554e6175)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:04.094686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:04.104791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:04.155887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ee4cacd5064113b7705630554e6175', 'request_id': 'req-c8636db7-8ef7-4c8c-a992-730e90bf6959'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.165377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ee4cacd5064113b7705630554e6175', 'request_id': 'req-c8636db7-8ef7-4c8c-a992-730e90bf6959'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.171499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ee4cacd5064113b7705630554e6175', 'request_id': 'req-c8636db7-8ef7-4c8c-a992-730e90bf6959'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.183801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] The token cache is being invalidate because project a6ee4cacd5064113b7705630554e6175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:04.184130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.186153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c8636db7-8ef7-4c8c-a992-730e90bf6959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.192068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1100/2250] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:04 2022] DELETE /identity/v3/projects/a6ee4cacd5064113b7705630554e6175 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:04.194600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6303ef-4a64-41fc-b217-b388236e4330 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.195087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6303ef-4a64-41fc-b217-b388236e4330 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.195682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6303ef-4a64-41fc-b217-b388236e4330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.226960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8f6303ef-4a64-41fc-b217-b388236e4330 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:04.228417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8f6303ef-4a64-41fc-b217-b388236e4330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.306524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1151/2251] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:04.314593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5fcd3-554a-4bb9-add6-751489fb7d0e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.315899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5fcd3-554a-4bb9-add6-751489fb7d0e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.316248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5fcd3-554a-4bb9-add6-751489fb7d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.338676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bcf5fcd3-554a-4bb9-add6-751489fb7d0e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:04.340042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bcf5fcd3-554a-4bb9-add6-751489fb7d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.349708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1101/2252] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:04.354567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6303ef-4a64-41fc-b217-b388236e4330 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:04.361998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5fcd3-554a-4bb9-add6-751489fb7d0e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:04.364264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bcf5fcd3-554a-4bb9-add6-751489fb7d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.370475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6303ef-4a64-41fc-b217-b388236e4330 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:04.376230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:04.380004 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5fcd3-554a-4bb9-add6-751489fb7d0e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:04.386201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.386572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.386931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.388187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:39:04.388187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:04.390858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:04.410376 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1152/2253] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:39:04.411552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:04.414818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.419922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:04.420648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.420939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.421221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] PATH_INFO: `/v3/projects/1ce064f4048447e89f4ae0ccb1d916a5` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.427518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce064f4048447e89f4ae0ccb1d916a5)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:04.429483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:04.438459 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb273c7f-dcf8-40dc-8b6c-898a281a8490 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:04.445839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:04.446610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.446897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.447178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] PATH_INFO: `/v3/projects/77d1810d277344d6b6aeeb65d134e618` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.448402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d1810d277344d6b6aeeb65d134e618)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:04.450382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:04.544178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d1810d277344d6b6aeeb65d134e618', 'request_id': 'req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.553514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d1810d277344d6b6aeeb65d134e618', 'request_id': 'req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.558158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce064f4048447e89f4ae0ccb1d916a5', 'request_id': 'req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.567020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d1810d277344d6b6aeeb65d134e618', 'request_id': 'req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.569446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:04.570242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce064f4048447e89f4ae0ccb1d916a5', 'request_id': 'req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.579165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce064f4048447e89f4ae0ccb1d916a5', 'request_id': 'req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.592159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] The token cache is being invalidate because project 77d1810d277344d6b6aeeb65d134e618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:04.592860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.595438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.598079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] The token cache is being invalidate because project 1ce064f4048447e89f4ae0ccb1d916a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:04.598572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.599631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1153/2254] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:04 2022] DELETE /identity/v3/projects/77d1810d277344d6b6aeeb65d134e618 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:04.600319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2f4dc2d2-99a5-489c-a4ad-8fc90697e9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.602153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:04.603481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1102/2255] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:04 2022] DELETE /identity/v3/projects/1ce064f4048447e89f4ae0ccb1d916a5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:04.606209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.608055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660578a-7f5d-4c3b-8923-4ae12b5d676c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.608379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660578a-7f5d-4c3b-8923-4ae12b5d676c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.608711 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660578a-7f5d-4c3b-8923-4ae12b5d676c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.635296 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c660578a-7f5d-4c3b-8923-4ae12b5d676c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:04.636555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c660578a-7f5d-4c3b-8923-4ae12b5d676c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.720804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1103/2256] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:04.720804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660578a-7f5d-4c3b-8923-4ae12b5d676c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:04.720804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c660578a-7f5d-4c3b-8923-4ae12b5d676c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.731305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660578a-7f5d-4c3b-8923-4ae12b5d676c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:04.736847 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:04.738304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.738644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.738928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] PATH_INFO: `/v3/users/437a35c22aa94d0eb8b1e487c6b47d6d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.740275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] RBAC: Authorizing `identity:delete_user(user_id=437a35c22aa94d0eb8b1e487c6b47d6d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:04.742409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:04.749276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:04.759793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbc270-ca45-46e5-9285-7f4c8fb38f55 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:04.767439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4c5bf-c0b4-4dd1-a956-035674720fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:04.768780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4c5bf-c0b4-4dd1-a956-035674720fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.769104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4c5bf-c0b4-4dd1-a956-035674720fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.769531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4c5bf-c0b4-4dd1-a956-035674720fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.770254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4c5bf-c0b4-4dd1-a956-035674720fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:04.773499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4c5bf-c0b4-4dd1-a956-035674720fc6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:04.806601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1154/2257] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:04 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:04.807133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437a35c22aa94d0eb8b1e487c6b47d6d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.807910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a93aaf-e5c1-4b88-a3d3-5714098bedb7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.808204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a93aaf-e5c1-4b88-a3d3-5714098bedb7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.808486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a93aaf-e5c1-4b88-a3d3-5714098bedb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.818623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437a35c22aa94d0eb8b1e487c6b47d6d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.820179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437a35c22aa94d0eb8b1e487c6b47d6d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.835825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-12a93aaf-e5c1-4b88-a3d3-5714098bedb7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:04.837324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-12a93aaf-e5c1-4b88-a3d3-5714098bedb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.852162 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437a35c22aa94d0eb8b1e487c6b47d6d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.853926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437a35c22aa94d0eb8b1e487c6b47d6d'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:04.872805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1104/2258] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:04 2022] DELETE /identity/v3/users/437a35c22aa94d0eb8b1e487c6b47d6d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:04.874028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:04.875667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:04.912876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1155/2259] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:04.917664 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a93aaf-e5c1-4b88-a3d3-5714098bedb7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:04.929743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a93aaf-e5c1-4b88-a3d3-5714098bedb7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:04.955076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6737e-c74c-492e-8e6b-f1cbaf16c30f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:04.961122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a256b-6534-419f-940b-354a48c4aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:04.962141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a256b-6534-419f-940b-354a48c4aaa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:04.962430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a256b-6534-419f-940b-354a48c4aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:04.962708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a256b-6534-419f-940b-354a48c4aaa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:04.963385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a256b-6534-419f-940b-354a48c4aaa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:04.977326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a256b-6534-419f-940b-354a48c4aaa7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.000597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1105/2260] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:04 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:05.006758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a256b-6534-419f-940b-354a48c4aaa7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.008826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.009569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.009850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.010134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.010836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:39:05.011141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.013022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.019540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a256b-6534-419f-940b-354a48c4aaa7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.025581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a5412-bdec-45be-8d0b-56c180ec7346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.026210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a5412-bdec-45be-8d0b-56c180ec7346 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.026492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a5412-bdec-45be-8d0b-56c180ec7346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.026775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a5412-bdec-45be-8d0b-56c180ec7346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.027712 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a5412-bdec-45be-8d0b-56c180ec7346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.027820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1156/2261] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:39:05.029113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a5412-bdec-45be-8d0b-56c180ec7346 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.031958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.040428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1106/2262] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:39:05.043837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fc995-e7c9-4167-81fc-ca53c7eff087 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.044436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a5412-bdec-45be-8d0b-56c180ec7346 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.048902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc850728-9d59-4424-9767-370e97e2ab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.049889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc850728-9d59-4424-9767-370e97e2ab9e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.049889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc850728-9d59-4424-9767-370e97e2ab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.050306 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc850728-9d59-4424-9767-370e97e2ab9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.050949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc850728-9d59-4424-9767-370e97e2ab9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.052744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc850728-9d59-4424-9767-370e97e2ab9e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.055873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a5412-bdec-45be-8d0b-56c180ec7346 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.061386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829e15b-ecb6-4ccd-b814-ad7cca37de34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.062088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829e15b-ecb6-4ccd-b814-ad7cca37de34 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.062456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829e15b-ecb6-4ccd-b814-ad7cca37de34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.062811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829e15b-ecb6-4ccd-b814-ad7cca37de34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.063928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829e15b-ecb6-4ccd-b814-ad7cca37de34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.066358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829e15b-ecb6-4ccd-b814-ad7cca37de34 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.066493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:05.084011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1107/2263] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:39:05.085041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829e15b-ecb6-4ccd-b814-ad7cca37de34 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.088943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1157/2264] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:39:05.093901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc850728-9d59-4424-9767-370e97e2ab9e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.103043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829e15b-ecb6-4ccd-b814-ad7cca37de34 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.160597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:05.175566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc850728-9d59-4424-9767-370e97e2ab9e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.176917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c9af9-8bea-40c6-a9f5-7098b7eaaf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9af9-8bea-40c6-a9f5-7098b7eaaf94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.178003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9af9-8bea-40c6-a9f5-7098b7eaaf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.178289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9af9-8bea-40c6-a9f5-7098b7eaaf94 admin admin] PATH_INFO: `/v3/projects/8ed57f125f844b4d9bda4b940007a39e/users/a4710f436b59417f917de71614959565/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.181764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1858c4e-e8df-4921-b077-b0ac3a630388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.182389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1858c4e-e8df-4921-b077-b0ac3a630388 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.182673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1858c4e-e8df-4921-b077-b0ac3a630388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.182946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1858c4e-e8df-4921-b077-b0ac3a630388 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.183640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1858c4e-e8df-4921-b077-b0ac3a630388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.185656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1858c4e-e8df-4921-b077-b0ac3a630388 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.195884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c9af9-8bea-40c6-a9f5-7098b7eaaf94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed57f125f844b4d9bda4b940007a39e, user_id=a4710f436b59417f917de71614959565, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.198608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c9af9-8bea-40c6-a9f5-7098b7eaaf94 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.212204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1158/2265] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:39:05.213767 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1108/2266] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:05 2022] PUT /identity/v3/projects/8ed57f125f844b4d9bda4b940007a39e/users/a4710f436b59417f917de71614959565/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:39:05.215909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1858c4e-e8df-4921-b077-b0ac3a630388 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.224620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3831b0-52b2-49da-bb2e-97768fe9f94c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.224920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3831b0-52b2-49da-bb2e-97768fe9f94c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.225204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3831b0-52b2-49da-bb2e-97768fe9f94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.226577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1858c4e-e8df-4921-b077-b0ac3a630388 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.266825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ba3831b0-52b2-49da-bb2e-97768fe9f94c None None] MFA Rules not processed for user `a4710f436b59417f917de71614959565`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:05.270206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ba3831b0-52b2-49da-bb2e-97768fe9f94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:05.328795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.329458 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.329740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.330014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.331042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.332693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.353443 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1159/2267] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:05.353443 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.358981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:05.376834 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1109/2268] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:05.377802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3831b0-52b2-49da-bb2e-97768fe9f94c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.389398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a09076-a81b-410e-af63-f004989529a4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.390686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3831b0-52b2-49da-bb2e-97768fe9f94c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.396342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.397053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.397373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.397509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a064e-3494-4f5d-866b-40b2d3d2871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.397654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] PATH_INFO: `/v3/projects/c1870121634d49daa0e365229fa42138` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.398394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a064e-3494-4f5d-866b-40b2d3d2871d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.398693 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a064e-3494-4f5d-866b-40b2d3d2871d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.398895 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1870121634d49daa0e365229fa42138)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.399008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a064e-3494-4f5d-866b-40b2d3d2871d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.399649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a064e-3494-4f5d-866b-40b2d3d2871d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.401106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.401238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a064e-3494-4f5d-866b-40b2d3d2871d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.424063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1110/2269] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:05.424862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a064e-3494-4f5d-866b-40b2d3d2871d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.442916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a064e-3494-4f5d-866b-40b2d3d2871d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.457044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c996f7c-fcee-4cf6-9372-ce86d4c7a7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.457749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996f7c-fcee-4cf6-9372-ce86d4c7a7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.458089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996f7c-fcee-4cf6-9372-ce86d4c7a7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.458727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996f7c-fcee-4cf6-9372-ce86d4c7a7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.459036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c996f7c-fcee-4cf6-9372-ce86d4c7a7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.460992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c996f7c-fcee-4cf6-9372-ce86d4c7a7b6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.488800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1870121634d49daa0e365229fa42138', 'request_id': 'req-27970a64-18e6-475b-ba4f-e79c02c276d1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:05.499115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1870121634d49daa0e365229fa42138', 'request_id': 'req-27970a64-18e6-475b-ba4f-e79c02c276d1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:05.500254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1111/2270] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:05.501073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c996f7c-fcee-4cf6-9372-ce86d4c7a7b6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.511839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1870121634d49daa0e365229fa42138', 'request_id': 'req-27970a64-18e6-475b-ba4f-e79c02c276d1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:05.525843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:05.529195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] The token cache is being invalidate because project c1870121634d49daa0e365229fa42138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:05.529516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:05.530397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:05.536230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1160/2271] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:05 2022] DELETE /identity/v3/projects/c1870121634d49daa0e365229fa42138 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:05.536474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.538673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:05.588525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c996f7c-fcee-4cf6-9372-ce86d4c7a7b6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.594032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee63fd-f18c-45bb-93f6-4d3628eb3f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.595024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee63fd-f18c-45bb-93f6-4d3628eb3f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.595375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee63fd-f18c-45bb-93f6-4d3628eb3f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.595912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee63fd-f18c-45bb-93f6-4d3628eb3f66 admin admin] PATH_INFO: `/v3/projects/e52da7e888bd498892ff4c4878b83da6/users/fe5a7da9323c4397bf44c3bd4d11bd28/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.621317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee63fd-f18c-45bb-93f6-4d3628eb3f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52da7e888bd498892ff4c4878b83da6, user_id=fe5a7da9323c4397bf44c3bd4d11bd28, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.629031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee63fd-f18c-45bb-93f6-4d3628eb3f66 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.639228 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27970a64-18e6-475b-ba4f-e79c02c276d1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.650613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1112/2272] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:05 2022] PUT /identity/v3/projects/e52da7e888bd498892ff4c4878b83da6/users/fe5a7da9323c4397bf44c3bd4d11bd28/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:05.651901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe781d3-34ff-410b-9980-bf725b852c3a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.652190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe781d3-34ff-410b-9980-bf725b852c3a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.652466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe781d3-34ff-410b-9980-bf725b852c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.677764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8fe781d3-34ff-410b-9980-bf725b852c3a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:05.678484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8fe781d3-34ff-410b-9980-bf725b852c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:05.691600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:05.757822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d88e93-1868-4f80-a4ce-d4f8db55b288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.758627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88e93-1868-4f80-a4ce-d4f8db55b288 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.758946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88e93-1868-4f80-a4ce-d4f8db55b288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.759252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88e93-1868-4f80-a4ce-d4f8db55b288 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.759980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d88e93-1868-4f80-a4ce-d4f8db55b288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.762966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d88e93-1868-4f80-a4ce-d4f8db55b288 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.777495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1113/2273] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:05.779620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61968d81-c4b0-4015-8d30-164805c3ec2f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.780275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61968d81-c4b0-4015-8d30-164805c3ec2f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.780850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61968d81-c4b0-4015-8d30-164805c3ec2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.798369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1161/2274] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:05.798369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3abe6-babc-4afb-9ad2-3d30059eb196 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.798369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3abe6-babc-4afb-9ad2-3d30059eb196 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.798369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3abe6-babc-4afb-9ad2-3d30059eb196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.820543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ebd3abe6-babc-4afb-9ad2-3d30059eb196 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:05.821798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-61968d81-c4b0-4015-8d30-164805c3ec2f None None] MFA Rules not processed for user `fe5a7da9323c4397bf44c3bd4d11bd28`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:05.821937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ebd3abe6-babc-4afb-9ad2-3d30059eb196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:05.823265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-61968d81-c4b0-4015-8d30-164805c3ec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:05.834311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1162/2275] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:05.834311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3abe6-babc-4afb-9ad2-3d30059eb196 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.843087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3abe6-babc-4afb-9ad2-3d30059eb196 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.853801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087d48c-7f30-4b4c-94a1-0a85d990c11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.854588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087d48c-7f30-4b4c-94a1-0a85d990c11f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.854890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087d48c-7f30-4b4c-94a1-0a85d990c11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.855344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087d48c-7f30-4b4c-94a1-0a85d990c11f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.856222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087d48c-7f30-4b4c-94a1-0a85d990c11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.858366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087d48c-7f30-4b4c-94a1-0a85d990c11f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.874425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1163/2276] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:05.874425 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087d48c-7f30-4b4c-94a1-0a85d990c11f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.886938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087d48c-7f30-4b4c-94a1-0a85d990c11f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.894809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.894809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.894809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.895149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.895657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:39:05.895969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.897763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.912659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1114/2277] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:05.913421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61968d81-c4b0-4015-8d30-164805c3ec2f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.917087 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1164/2278] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:39:05.921092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.925184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61968d81-c4b0-4015-8d30-164805c3ec2f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.931054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.931682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.931967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.932273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.932987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.933797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe03e8-30f1-4699-af06-99fb2f900d9d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:05.934635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.940833 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a6224-d8e7-4af2-9e73-2a62932974eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:05.941738 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6224-d8e7-4af2-9e73-2a62932974eb admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:05.942093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6224-d8e7-4af2-9e73-2a62932974eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:05.942363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6224-d8e7-4af2-9e73-2a62932974eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:05.943035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a6224-d8e7-4af2-9e73-2a62932974eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:05.945244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a6224-d8e7-4af2-9e73-2a62932974eb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:05.950544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1115/2279] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:05.950544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:05.950544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:05.989119 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1165/2280] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:05 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:05.990112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a6224-d8e7-4af2-9e73-2a62932974eb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.065994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723ee1f-0fc0-4673-9b0d-ce9e56d58634 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.071193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:06.121323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a6224-d8e7-4af2-9e73-2a62932974eb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.128070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd5892-ce08-4030-80b8-7d227cca78e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.128796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd5892-ce08-4030-80b8-7d227cca78e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.129247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd5892-ce08-4030-80b8-7d227cca78e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.129695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd5892-ce08-4030-80b8-7d227cca78e5 admin admin] PATH_INFO: `/v3/projects/6e877532c550428ebbcb3fc718619d94/users/8a83d01e399e43bea594a9495bbaeb92/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.139827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:06.148576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf6d11-5d9c-4813-9170-302410cc5b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.149815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf6d11-5d9c-4813-9170-302410cc5b29 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.149944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd5892-ce08-4030-80b8-7d227cca78e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e877532c550428ebbcb3fc718619d94, user_id=8a83d01e399e43bea594a9495bbaeb92, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.150435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf6d11-5d9c-4813-9170-302410cc5b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.150903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf6d11-5d9c-4813-9170-302410cc5b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.152750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd5892-ce08-4030-80b8-7d227cca78e5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.182904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1166/2281] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:05 2022] PUT /identity/v3/projects/6e877532c550428ebbcb3fc718619d94/users/8a83d01e399e43bea594a9495bbaeb92/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:06.184723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd5892-ce08-4030-80b8-7d227cca78e5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.230370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf6d11-5d9c-4813-9170-302410cc5b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.232388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf6d11-5d9c-4813-9170-302410cc5b29 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.246468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1116/2282] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:06.248647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a52aab-268e-4aa0-840d-cd9d2914f2b7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.249141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a52aab-268e-4aa0-840d-cd9d2914f2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.249634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a52aab-268e-4aa0-840d-cd9d2914f2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.269715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd5892-ce08-4030-80b8-7d227cca78e5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.276369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca26b81-5060-4320-8862-47ebfcf85a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.277007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca26b81-5060-4320-8862-47ebfcf85a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.277287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca26b81-5060-4320-8862-47ebfcf85a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.277592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca26b81-5060-4320-8862-47ebfcf85a3b admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.278321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca26b81-5060-4320-8862-47ebfcf85a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.280261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca26b81-5060-4320-8862-47ebfcf85a3b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.287313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c3a52aab-268e-4aa0-840d-cd9d2914f2b7 None None] MFA Rules not processed for user `8a83d01e399e43bea594a9495bbaeb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:06.289161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c3a52aab-268e-4aa0-840d-cd9d2914f2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:06.311177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1167/2283] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:06 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:06.315108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca26b81-5060-4320-8862-47ebfcf85a3b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.325258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca26b81-5060-4320-8862-47ebfcf85a3b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.330843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ffcf3-8538-404d-93ed-4992a537d15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.331475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ffcf3-8538-404d-93ed-4992a537d15c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.331754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ffcf3-8538-404d-93ed-4992a537d15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.332028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ffcf3-8538-404d-93ed-4992a537d15c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.332737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ffcf3-8538-404d-93ed-4992a537d15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.334476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ffcf3-8538-404d-93ed-4992a537d15c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.348823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1168/2284] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:39:06.349885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ffcf3-8538-404d-93ed-4992a537d15c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.360412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ffcf3-8538-404d-93ed-4992a537d15c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.366047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051761b-9025-4079-b86c-97841b4f2ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.366673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051761b-9025-4079-b86c-97841b4f2ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.367388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051761b-9025-4079-b86c-97841b4f2ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.367665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051761b-9025-4079-b86c-97841b4f2ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.368642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051761b-9025-4079-b86c-97841b4f2ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.371072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051761b-9025-4079-b86c-97841b4f2ff7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.388133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1169/2285] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:06.389570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051761b-9025-4079-b86c-97841b4f2ff7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.392342 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1117/2286] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:06.397649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a52aab-268e-4aa0-840d-cd9d2914f2b7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.403943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051761b-9025-4079-b86c-97841b4f2ff7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.419551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febe855-553d-48da-9e1d-e75a77a98ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.420763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febe855-553d-48da-9e1d-e75a77a98ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.421319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febe855-553d-48da-9e1d-e75a77a98ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.421985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febe855-553d-48da-9e1d-e75a77a98ec1 admin admin] PATH_INFO: `/v3/projects/5c6c623d52f64907bf414404d6ad9587/users/393698be4fc54587aa8ba3cbf1b8fac9/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.434774 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a52aab-268e-4aa0-840d-cd9d2914f2b7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.446405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a3679-c9ec-4e36-b171-82878bae12c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.447053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3679-c9ec-4e36-b171-82878bae12c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.447340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3679-c9ec-4e36-b171-82878bae12c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.447613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3679-c9ec-4e36-b171-82878bae12c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.453635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a3679-c9ec-4e36-b171-82878bae12c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.453954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febe855-553d-48da-9e1d-e75a77a98ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6c623d52f64907bf414404d6ad9587, user_id=393698be4fc54587aa8ba3cbf1b8fac9, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.454061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febe855-553d-48da-9e1d-e75a77a98ec1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.458483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a3679-c9ec-4e36-b171-82878bae12c5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.491516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1170/2287] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:06 2022] PUT /identity/v3/projects/5c6c623d52f64907bf414404d6ad9587/users/393698be4fc54587aa8ba3cbf1b8fac9/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:06.492681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038c113-73b4-49bd-a0f3-7c8f0bdea5aa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.492962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038c113-73b4-49bd-a0f3-7c8f0bdea5aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.493359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038c113-73b4-49bd-a0f3-7c8f0bdea5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.516389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1118/2288] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:06 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:06.522927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a3679-c9ec-4e36-b171-82878bae12c5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.534402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2038c113-73b4-49bd-a0f3-7c8f0bdea5aa None None] MFA Rules not processed for user `393698be4fc54587aa8ba3cbf1b8fac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:06.535691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2038c113-73b4-49bd-a0f3-7c8f0bdea5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:06.641484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a3679-c9ec-4e36-b171-82878bae12c5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.650649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9120f6d-a960-48a6-9ca8-6c7bc71ea4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.652308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9120f6d-a960-48a6-9ca8-6c7bc71ea4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.652451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9120f6d-a960-48a6-9ca8-6c7bc71ea4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.652917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9120f6d-a960-48a6-9ca8-6c7bc71ea4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.654303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9120f6d-a960-48a6-9ca8-6c7bc71ea4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.654869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:06.656494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9120f6d-a960-48a6-9ca8-6c7bc71ea4c4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.661496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1171/2289] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:06.677059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038c113-73b4-49bd-a0f3-7c8f0bdea5aa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.680305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:06.691649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1119/2290] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:06 2022] POST /identity/v3/users => generated 296 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:39:06.692522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9120f6d-a960-48a6-9ca8-6c7bc71ea4c4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.710397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9120f6d-a960-48a6-9ca8-6c7bc71ea4c4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.717541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec854c7e-626a-48ba-ab24-8400be4fbb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.718453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec854c7e-626a-48ba-ab24-8400be4fbb71 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.718771 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec854c7e-626a-48ba-ab24-8400be4fbb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.719143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec854c7e-626a-48ba-ab24-8400be4fbb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.720195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec854c7e-626a-48ba-ab24-8400be4fbb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.723208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec854c7e-626a-48ba-ab24-8400be4fbb71 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.739186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1120/2291] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:06.743204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec854c7e-626a-48ba-ab24-8400be4fbb71 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.768315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec854c7e-626a-48ba-ab24-8400be4fbb71 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.780072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038c113-73b4-49bd-a0f3-7c8f0bdea5aa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.783186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f72e81-bf64-41f6-8fc5-539f5c5ea5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.785110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f72e81-bf64-41f6-8fc5-539f5c5ea5c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.786696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f72e81-bf64-41f6-8fc5-539f5c5ea5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.786696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f72e81-bf64-41f6-8fc5-539f5c5ea5c9 admin admin] PATH_INFO: `/v3/projects/ee176ef36a48472b93b64f838feac40a/users/08d4b036660e48009879c0fa78b186c3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.790897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c304da-3ee9-4d8f-a06b-f5908d8864b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.792403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c304da-3ee9-4d8f-a06b-f5908d8864b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.792966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c304da-3ee9-4d8f-a06b-f5908d8864b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.793544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c304da-3ee9-4d8f-a06b-f5908d8864b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.820596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c304da-3ee9-4d8f-a06b-f5908d8864b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.822912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c304da-3ee9-4d8f-a06b-f5908d8864b2 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.835061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f72e81-bf64-41f6-8fc5-539f5c5ea5c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee176ef36a48472b93b64f838feac40a, user_id=08d4b036660e48009879c0fa78b186c3, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.837239 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1172/2292] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:39:06.838576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f72e81-bf64-41f6-8fc5-539f5c5ea5c9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.865469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1121/2293] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:06 2022] PUT /identity/v3/projects/ee176ef36a48472b93b64f838feac40a/users/08d4b036660e48009879c0fa78b186c3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:06.866461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c304da-3ee9-4d8f-a06b-f5908d8864b2 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:06.883351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c304da-3ee9-4d8f-a06b-f5908d8864b2 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:06.960813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14e02a-3696-45ae-9020-9c0bebde0c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:06.961608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e02a-3696-45ae-9020-9c0bebde0c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:06.961960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e02a-3696-45ae-9020-9c0bebde0c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:06.962421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e02a-3696-45ae-9020-9c0bebde0c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:06.963591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14e02a-3696-45ae-9020-9c0bebde0c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:06.966574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14e02a-3696-45ae-9020-9c0bebde0c70 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:06.978814 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1173/2294] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:39:06.991591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f72e81-bf64-41f6-8fc5-539f5c5ea5c9 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:07.011795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f72e81-bf64-41f6-8fc5-539f5c5ea5c9 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:07.017631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:07.019289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:07.019491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:07.019773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] PATH_INFO: `/v3/projects/ee176ef36a48472b93b64f838feac40a/users/08d4b036660e48009879c0fa78b186c3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:07.026707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee176ef36a48472b93b64f838feac40a, user_id=08d4b036660e48009879c0fa78b186c3, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:07.030806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:07.056681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1122/2295] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:06 2022] PUT /identity/v3/projects/ee176ef36a48472b93b64f838feac40a/users/08d4b036660e48009879c0fa78b186c3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:07.058363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d47d60-e4ef-486c-8065-dbf3ab357875 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:07.058501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d47d60-e4ef-486c-8065-dbf3ab357875 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:07.058804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d47d60-e4ef-486c-8065-dbf3ab357875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:07.249705 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-73d47d60-e4ef-486c-8065-dbf3ab357875 None None] MFA Rules not processed for user `08d4b036660e48009879c0fa78b186c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:07.250991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-73d47d60-e4ef-486c-8065-dbf3ab357875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:07.409014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1174/2296] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:07.431082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:07.431082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:07.520248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40319ea6-5c0b-466a-a348-1ce86d7f4068 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:07.524078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6db5b-49e8-41e7-9610-709c74fb6678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:07.524700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6db5b-49e8-41e7-9610-709c74fb6678 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:07.524981 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6db5b-49e8-41e7-9610-709c74fb6678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:07.525253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6db5b-49e8-41e7-9610-709c74fb6678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:07.538558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6db5b-49e8-41e7-9610-709c74fb6678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:07.540046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6db5b-49e8-41e7-9610-709c74fb6678 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:07.552098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1123/2297] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:07.600214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d47d60-e4ef-486c-8065-dbf3ab357875 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:07.614024 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d47d60-e4ef-486c-8065-dbf3ab357875 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:07.617726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6adcb5-e54b-4b4b-930c-1c2c777695c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:07.618411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6adcb5-e54b-4b4b-930c-1c2c777695c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:07.618694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6adcb5-e54b-4b4b-930c-1c2c777695c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:07.618963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6adcb5-e54b-4b4b-930c-1c2c777695c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:07.620697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1c6adcb5-e54b-4b4b-930c-1c2c777695c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:07.642509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:07.689864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6adcb5-e54b-4b4b-930c-1c2c777695c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:07.691594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6adcb5-e54b-4b4b-930c-1c2c777695c8 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:07.702668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1175/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:11.769764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6db5b-49e8-41e7-9610-709c74fb6678 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:11.835680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6db5b-49e8-41e7-9610-709c74fb6678 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:11.840317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:11.840961 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:11.841247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:11.841552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] PATH_INFO: `/v3/users/a4710f436b59417f917de71614959565` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:11.842719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4710f436b59417f917de71614959565)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:11.844757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:11.896624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4710f436b59417f917de71614959565'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:11.909969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4710f436b59417f917de71614959565'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:11.910687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4710f436b59417f917de71614959565'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:11.913346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:11.926621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4710f436b59417f917de71614959565'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:11.927337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4710f436b59417f917de71614959565'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:11.939307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1124/2299] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:11 2022] DELETE /identity/v3/users/a4710f436b59417f917de71614959565 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:11.944753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b5aa5-a4e6-4730-9d1b-fb1b861e2096 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:11.945263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b5aa5-a4e6-4730-9d1b-fb1b861e2096 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:11.945378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b5aa5-a4e6-4730-9d1b-fb1b861e2096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:11.964422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ca1b5aa5-a4e6-4730-9d1b-fb1b861e2096 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:11.965675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ca1b5aa5-a4e6-4730-9d1b-fb1b861e2096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:12.025572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1176/2300] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:12.035387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:12.036765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:12.097124 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75cd87-6fb2-414a-9d55-c661398c0319 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:12.101804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad0fca-277a-4e3a-bea5-6716ca6600aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:12.102418 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad0fca-277a-4e3a-bea5-6716ca6600aa service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:12.102804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad0fca-277a-4e3a-bea5-6716ca6600aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:12.103182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad0fca-277a-4e3a-bea5-6716ca6600aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:12.114789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad0fca-277a-4e3a-bea5-6716ca6600aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:12.116328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad0fca-277a-4e3a-bea5-6716ca6600aa service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:12.126061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1125/2301] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:12.420938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b5aa5-a4e6-4730-9d1b-fb1b861e2096 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:12.422407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ca1b5aa5-a4e6-4730-9d1b-fb1b861e2096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:12.438896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b5aa5-a4e6-4730-9d1b-fb1b861e2096 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:12.458667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:12.458848 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:12.459130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:12.459386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] PATH_INFO: `/v3/users/393698be4fc54587aa8ba3cbf1b8fac9` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:12.460555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=393698be4fc54587aa8ba3cbf1b8fac9)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:12.462516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:12.529784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '393698be4fc54587aa8ba3cbf1b8fac9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.539772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '393698be4fc54587aa8ba3cbf1b8fac9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.548846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '393698be4fc54587aa8ba3cbf1b8fac9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.550208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '393698be4fc54587aa8ba3cbf1b8fac9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.569644 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '393698be4fc54587aa8ba3cbf1b8fac9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.577874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1177/2302] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:12 2022] DELETE /identity/v3/users/393698be4fc54587aa8ba3cbf1b8fac9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:12.584366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34d5c6-2acd-43cc-be98-016d78c6a7f7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:12.585130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34d5c6-2acd-43cc-be98-016d78c6a7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:12.585625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34d5c6-2acd-43cc-be98-016d78c6a7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:12.592965 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:12.599643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:12.623938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ff34d5c6-2acd-43cc-be98-016d78c6a7f7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:12.625178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ff34d5c6-2acd-43cc-be98-016d78c6a7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:12.658742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:12.702519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478363-feec-4232-bf6e-dd608bfcdd28 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:12.708682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:12.709430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:12.709810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:12.710246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1126/2303] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:12.710399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] PATH_INFO: `/v3/projects/8ed57f125f844b4d9bda4b940007a39e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:12.711556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed57f125f844b4d9bda4b940007a39e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:12.713904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:12.729547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34d5c6-2acd-43cc-be98-016d78c6a7f7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:12.729547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ff34d5c6-2acd-43cc-be98-016d78c6a7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:12.771902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed57f125f844b4d9bda4b940007a39e', 'request_id': 'req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.777816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed57f125f844b4d9bda4b940007a39e', 'request_id': 'req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.787413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed57f125f844b4d9bda4b940007a39e', 'request_id': 'req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.797542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34d5c6-2acd-43cc-be98-016d78c6a7f7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:12.799650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] The token cache is being invalidate because project 8ed57f125f844b4d9bda4b940007a39e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:12.800149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.800799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.802564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1178/2304] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:12 2022] DELETE /identity/v3/projects/8ed57f125f844b4d9bda4b940007a39e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:12.806797 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:12.808005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:12.852292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:12.877606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208ddc3-6ea2-4b6a-970b-913b9edf86f8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:12.880618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:12.881490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:12.881928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:12.882570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:12.882806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:12.883512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:12.883863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:12.884206 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] PATH_INFO: `/v3/users/8a83d01e399e43bea594a9495bbaeb92` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:12.885476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:12.885696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a83d01e399e43bea594a9495bbaeb92)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:12.888045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:12.903176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:12.905142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:12.929481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1127/2305] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:39:12.963426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a83d01e399e43bea594a9495bbaeb92'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.973331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a83d01e399e43bea594a9495bbaeb92'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.986946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a83d01e399e43bea594a9495bbaeb92'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.988051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a83d01e399e43bea594a9495bbaeb92'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:12.995153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:13.010623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a83d01e399e43bea594a9495bbaeb92'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.023734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1179/2306] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:12 2022] DELETE /identity/v3/users/8a83d01e399e43bea594a9495bbaeb92 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:39:13.343108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:13.344203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:13.357582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2c10c-7882-499f-b419-eefabe7837dd service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:13.445447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:13.447582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:13.498983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:13.500485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:13.503508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:13.504996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] PATH_INFO: `/v3/projects/5c6c623d52f64907bf414404d6ad9587` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:13.506741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6c623d52f64907bf414404d6ad9587)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:13.509069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:13.568108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cf7f4-cc12-460f-b38c-2ce5fcd22c39 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:13.577691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:13.578623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:13.578982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:13.579335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] PATH_INFO: `/v3/projects/6e877532c550428ebbcb3fc718619d94` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:13.581095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e877532c550428ebbcb3fc718619d94)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:13.583383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:13.606708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6c623d52f64907bf414404d6ad9587', 'request_id': 'req-f4deda81-48ae-43fc-8908-f24337ab7245'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.617763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6c623d52f64907bf414404d6ad9587', 'request_id': 'req-f4deda81-48ae-43fc-8908-f24337ab7245'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.626550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6c623d52f64907bf414404d6ad9587', 'request_id': 'req-f4deda81-48ae-43fc-8908-f24337ab7245'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.646185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] The token cache is being invalidate because project 5c6c623d52f64907bf414404d6ad9587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:13.646404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.647448 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f4deda81-48ae-43fc-8908-f24337ab7245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.650396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1128/2307] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:13 2022] DELETE /identity/v3/projects/5c6c623d52f64907bf414404d6ad9587 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:13.660936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e410cd-4e04-4dc0-94cf-16e8065930b5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:13.661367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e410cd-4e04-4dc0-94cf-16e8065930b5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:13.661739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e410cd-4e04-4dc0-94cf-16e8065930b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:13.697811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c0e410cd-4e04-4dc0-94cf-16e8065930b5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:13.699213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c0e410cd-4e04-4dc0-94cf-16e8065930b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:13.704838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e877532c550428ebbcb3fc718619d94', 'request_id': 'req-2939c254-6a56-4e7b-83c4-ccc949c4233a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.716326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e877532c550428ebbcb3fc718619d94', 'request_id': 'req-2939c254-6a56-4e7b-83c4-ccc949c4233a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.744098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e877532c550428ebbcb3fc718619d94', 'request_id': 'req-2939c254-6a56-4e7b-83c4-ccc949c4233a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.762219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] The token cache is being invalidate because project 6e877532c550428ebbcb3fc718619d94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:13.762642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.763404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.765808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1180/2308] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:13 2022] DELETE /identity/v3/projects/6e877532c550428ebbcb3fc718619d94 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:13.771210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:13.772725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:13.858069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1129/2309] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:13.871748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ad5e7-cefc-4bb3-ad24-369c60f36912 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:13.872422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ad5e7-cefc-4bb3-ad24-369c60f36912 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:13.872918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ad5e7-cefc-4bb3-ad24-369c60f36912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:13.880760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939c254-6a56-4e7b-83c4-ccc949c4233a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:13.886800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:13.887790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:13.888279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:13.888790 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] PATH_INFO: `/v3/users/08d4b036660e48009879c0fa78b186c3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:13.890593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d4b036660e48009879c0fa78b186c3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:13.894178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:13.894848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:13.903306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-591ad5e7-cefc-4bb3-ad24-369c60f36912 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:13.906191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-591ad5e7-cefc-4bb3-ad24-369c60f36912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:13.915763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:13.916948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1130/2310] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:13.921413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ad5e7-cefc-4bb3-ad24-369c60f36912 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:13.932226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ad5e7-cefc-4bb3-ad24-369c60f36912 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:13.937427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:13.938112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:13.938463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:13.938833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:13.940044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:39:13.940152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:13.942000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:13.958652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1131/2311] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:39:13.959997 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:13.960383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d4b036660e48009879c0fa78b186c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.970042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d4b036660e48009879c0fa78b186c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.971743 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00fd36-b82b-4222-a36f-bc0c20ffc81e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:13.977947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2efcc1-62cf-4ed4-b389-c788efaef03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:13.978748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efcc1-62cf-4ed4-b389-c788efaef03f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:13.978971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efcc1-62cf-4ed4-b389-c788efaef03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:13.979266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efcc1-62cf-4ed4-b389-c788efaef03f admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:13.979501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d4b036660e48009879c0fa78b186c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.980218 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2efcc1-62cf-4ed4-b389-c788efaef03f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:13.980382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d4b036660e48009879c0fa78b186c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:13.982312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2efcc1-62cf-4ed4-b389-c788efaef03f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.003712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d4b036660e48009879c0fa78b186c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:14.022162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1181/2312] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:13 2022] DELETE /identity/v3/users/08d4b036660e48009879c0fa78b186c3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:14.023292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1132/2313] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:13 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:14.027288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.028743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:14.113597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76d559-4289-4d86-b874-96cea3f9f5d8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.183610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e70ace-2383-44cf-820b-4561a9ae68da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.183610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e70ace-2383-44cf-820b-4561a9ae68da admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.183610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e70ace-2383-44cf-820b-4561a9ae68da admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.183610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e70ace-2383-44cf-820b-4561a9ae68da admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.184410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e70ace-2383-44cf-820b-4561a9ae68da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.184934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e70ace-2383-44cf-820b-4561a9ae68da admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.207947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1182/2314] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:14 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:14.211833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2efcc1-62cf-4ed4-b389-c788efaef03f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.223193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2efcc1-62cf-4ed4-b389-c788efaef03f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.229275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8204c-c856-497a-8630-7cba04d79c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.229937 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8204c-c856-497a-8630-7cba04d79c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.230223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8204c-c856-497a-8630-7cba04d79c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.230503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8204c-c856-497a-8630-7cba04d79c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.231211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8204c-c856-497a-8630-7cba04d79c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.232885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8204c-c856-497a-8630-7cba04d79c9d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.245089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1133/2315] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:39:14.249125 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e70ace-2383-44cf-820b-4561a9ae68da admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.261028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e70ace-2383-44cf-820b-4561a9ae68da admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.265792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.266329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.266606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.266876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.267778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.269430 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.283668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1183/2316] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:14.285417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8204c-c856-497a-8630-7cba04d79c9d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.296379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8204c-c856-497a-8630-7cba04d79c9d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.302327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2dea5b-1238-4f03-93cf-dbfc422b4395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.303007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dea5b-1238-4f03-93cf-dbfc422b4395 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.303373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dea5b-1238-4f03-93cf-dbfc422b4395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.303625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dea5b-1238-4f03-93cf-dbfc422b4395 admin admin] PATH_INFO: `/v3/projects/6b7e4c91aaaf48e2b134128055919a47/users/93dacbaabf7343bbae337761b8d8cd41/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.322307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2dea5b-1238-4f03-93cf-dbfc422b4395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b7e4c91aaaf48e2b134128055919a47, user_id=93dacbaabf7343bbae337761b8d8cd41, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.325018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2dea5b-1238-4f03-93cf-dbfc422b4395 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.329461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.331789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:14.340402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1134/2317] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:14 2022] PUT /identity/v3/projects/6b7e4c91aaaf48e2b134128055919a47/users/93dacbaabf7343bbae337761b8d8cd41/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:39:14.345082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b84ce-0df1-4fcb-90d4-f89ece9930f3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.345409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b84ce-0df1-4fcb-90d4-f89ece9930f3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.346555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b84ce-0df1-4fcb-90d4-f89ece9930f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.347183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae06eb-5806-4013-9311-6706ff4896fd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.365496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4d8b84ce-0df1-4fcb-90d4-f89ece9930f3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:14.366687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:14.366804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4d8b84ce-0df1-4fcb-90d4-f89ece9930f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:14.410552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.411197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.411475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.411745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] PATH_INFO: `/v3/projects/ee176ef36a48472b93b64f838feac40a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.413018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee176ef36a48472b93b64f838feac40a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.414962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.428944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1135/2318] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:14.440716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b84ce-0df1-4fcb-90d4-f89ece9930f3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.442143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4d8b84ce-0df1-4fcb-90d4-f89ece9930f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:14.483156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee176ef36a48472b93b64f838feac40a', 'request_id': 'req-032729f8-e8c1-48c0-ba34-2aee757a4170'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:14.489020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee176ef36a48472b93b64f838feac40a', 'request_id': 'req-032729f8-e8c1-48c0-ba34-2aee757a4170'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:14.498166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee176ef36a48472b93b64f838feac40a', 'request_id': 'req-032729f8-e8c1-48c0-ba34-2aee757a4170'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:14.510331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] The token cache is being invalidate because project ee176ef36a48472b93b64f838feac40a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:14.510622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:14.511265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-032729f8-e8c1-48c0-ba34-2aee757a4170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:14.513059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1184/2319] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:14 2022] DELETE /identity/v3/projects/ee176ef36a48472b93b64f838feac40a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:39:14.514106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b84ce-0df1-4fcb-90d4-f89ece9930f3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.519548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342332-5650-4fd6-8977-f66c195f4cba None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.519970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342332-5650-4fd6-8977-f66c195f4cba None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.520254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342332-5650-4fd6-8977-f66c195f4cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.540055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-99342332-5650-4fd6-8977-f66c195f4cba None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:14.541285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-99342332-5650-4fd6-8977-f66c195f4cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:14.559166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:14.566171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.566972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.567330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.567606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.569729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:14.611940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1185/2320] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:14.620724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc2483-b5ab-4e15-b79f-e8e3c5bf15c3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.621030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc2483-b5ab-4e15-b79f-e8e3c5bf15c3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.622116 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc2483-b5ab-4e15-b79f-e8e3c5bf15c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.644351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e9fc2483-b5ab-4e15-b79f-e8e3c5bf15c3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:14.645353 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.645496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e9fc2483-b5ab-4e15-b79f-e8e3c5bf15c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:14.646825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.653470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1186/2321] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:14.657468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1136/2322] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:39:14.657707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc2483-b5ab-4e15-b79f-e8e3c5bf15c3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.668188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc2483-b5ab-4e15-b79f-e8e3c5bf15c3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.674188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.674865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.675150 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.675485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.676200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:39:14.676549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.678337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.692296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1187/2323] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:39:14.696929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.709123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb568f77-dda4-46b3-aa3a-1b76ca1b927a service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.715841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0461-b86a-48a0-9afc-358f3598f1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.715841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0461-b86a-48a0-9afc-358f3598f1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.716063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0461-b86a-48a0-9afc-358f3598f1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.716424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0461-b86a-48a0-9afc-358f3598f1b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.717354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac0461-b86a-48a0-9afc-358f3598f1b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.719410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac0461-b86a-48a0-9afc-358f3598f1b5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.769674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1137/2324] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:14 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:14.773824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.788631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b38fe9-be7f-4f76-8080-f1aa439e50c1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.841627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:14.884672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0b6c4-0ba3-413d-bfba-d175a874bc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.885569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0b6c4-0ba3-413d-bfba-d175a874bc34 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.886136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0b6c4-0ba3-413d-bfba-d175a874bc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.886136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0b6c4-0ba3-413d-bfba-d175a874bc34 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.887750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0b6c4-0ba3-413d-bfba-d175a874bc34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.893572 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0b6c4-0ba3-413d-bfba-d175a874bc34 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.937138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1188/2325] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:14 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:14.946622 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0461-b86a-48a0-9afc-358f3598f1b5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.961089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0461-b86a-48a0-9afc-358f3598f1b5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:14.966045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597c803-f12b-4382-9425-5030f1f3ab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:14.966768 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597c803-f12b-4382-9425-5030f1f3ab44 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:14.967058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597c803-f12b-4382-9425-5030f1f3ab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:14.967331 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597c803-f12b-4382-9425-5030f1f3ab44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:14.968025 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597c803-f12b-4382-9425-5030f1f3ab44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:14.969675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597c803-f12b-4382-9425-5030f1f3ab44 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:14.982383 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1138/2326] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:14.986238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0b6c4-0ba3-413d-bfba-d175a874bc34 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:14.999024 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0b6c4-0ba3-413d-bfba-d175a874bc34 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:15.005102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c99c4-4411-4be4-b385-b61b6e68feea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:15.005930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c99c4-4411-4be4-b385-b61b6e68feea admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.006566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c99c4-4411-4be4-b385-b61b6e68feea admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.006566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c99c4-4411-4be4-b385-b61b6e68feea admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:15.007371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c99c4-4411-4be4-b385-b61b6e68feea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:15.010004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c99c4-4411-4be4-b385-b61b6e68feea admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:15.024469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1189/2327] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:15.028483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597c803-f12b-4382-9425-5030f1f3ab44 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:15.041606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597c803-f12b-4382-9425-5030f1f3ab44 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:15.051886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c2d6d-fb7a-44c1-9337-12e50d118fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:15.052858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2d6d-fb7a-44c1-9337-12e50d118fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.054046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2d6d-fb7a-44c1-9337-12e50d118fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.054359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2d6d-fb7a-44c1-9337-12e50d118fb0 admin admin] PATH_INFO: `/v3/projects/c44e432e64504eefb5f8ae47bb679270/users/7f9f6b845bd04e8bb50d7c554c004995/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:15.094790 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c2d6d-fb7a-44c1-9337-12e50d118fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c44e432e64504eefb5f8ae47bb679270, user_id=7f9f6b845bd04e8bb50d7c554c004995, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:15.100636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c2d6d-fb7a-44c1-9337-12e50d118fb0 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:15.137426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1139/2328] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:15 2022] PUT /identity/v3/projects/c44e432e64504eefb5f8ae47bb679270/users/7f9f6b845bd04e8bb50d7c554c004995/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:15.206531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcfc76-9ee3-49a9-87b0-2a76864b70e0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.206829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcfc76-9ee3-49a9-87b0-2a76864b70e0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.207107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcfc76-9ee3-49a9-87b0-2a76864b70e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:15.242613 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-27fcfc76-9ee3-49a9-87b0-2a76864b70e0 None None] MFA Rules not processed for user `7f9f6b845bd04e8bb50d7c554c004995`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:15.245389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-27fcfc76-9ee3-49a9-87b0-2a76864b70e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:15.391540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1190/2329] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:15.395858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c2d6d-fb7a-44c1-9337-12e50d118fb0 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:15.493246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c2d6d-fb7a-44c1-9337-12e50d118fb0 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:15.499750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8ecf1-2994-437c-ba85-fcaedec6dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:15.500550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ecf1-2994-437c-ba85-fcaedec6dce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.501009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ecf1-2994-437c-ba85-fcaedec6dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.501367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ecf1-2994-437c-ba85-fcaedec6dce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:15.503146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8ecf1-2994-437c-ba85-fcaedec6dce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:15.505951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8ecf1-2994-437c-ba85-fcaedec6dce1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:15.560201 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1140/2330] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:15 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:15.563895 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcfc76-9ee3-49a9-87b0-2a76864b70e0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:15.579657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcfc76-9ee3-49a9-87b0-2a76864b70e0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:15.651053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868e229-ffb5-4c72-a8e4-d817759006c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:15.651850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868e229-ffb5-4c72-a8e4-d817759006c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.652156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868e229-ffb5-4c72-a8e4-d817759006c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.652563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868e229-ffb5-4c72-a8e4-d817759006c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:15.653384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868e229-ffb5-4c72-a8e4-d817759006c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:15.655445 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868e229-ffb5-4c72-a8e4-d817759006c1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:15.691847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1191/2331] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:15 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:15.696196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8ecf1-2994-437c-ba85-fcaedec6dce1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:15.710726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8ecf1-2994-437c-ba85-fcaedec6dce1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:15.717040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88801928-6492-4788-bf6b-923292e49e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:15.717720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88801928-6492-4788-bf6b-923292e49e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.718009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88801928-6492-4788-bf6b-923292e49e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.718287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88801928-6492-4788-bf6b-923292e49e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:15.719011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88801928-6492-4788-bf6b-923292e49e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:15.721752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88801928-6492-4788-bf6b-923292e49e97 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:15.734953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1141/2332] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:15.739036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868e229-ffb5-4c72-a8e4-d817759006c1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:15.758343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868e229-ffb5-4c72-a8e4-d817759006c1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:15.763865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286755-2942-4e7c-a151-29c4a6fa450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:15.764567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286755-2942-4e7c-a151-29c4a6fa450e admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.764860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286755-2942-4e7c-a151-29c4a6fa450e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.765331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286755-2942-4e7c-a151-29c4a6fa450e admin admin] PATH_INFO: `/v3/projects/031e8090f49742eb9d7662cd74114622/users/92dca78daf8a435aa9214d2672750297/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:15.784078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286755-2942-4e7c-a151-29c4a6fa450e admin admin] RBAC: Authorizing `identity:create_grant(project_id=031e8090f49742eb9d7662cd74114622, user_id=92dca78daf8a435aa9214d2672750297, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:15.787489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286755-2942-4e7c-a151-29c4a6fa450e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:15.806356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1192/2333] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:15 2022] PUT /identity/v3/projects/031e8090f49742eb9d7662cd74114622/users/92dca78daf8a435aa9214d2672750297/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:15.810717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88801928-6492-4788-bf6b-923292e49e97 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:15.824929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88801928-6492-4788-bf6b-923292e49e97 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:15.836146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:15.922415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0fe7c-5412-4336-b552-867c52e35f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:15.923079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0fe7c-5412-4336-b552-867c52e35f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.923364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0fe7c-5412-4336-b552-867c52e35f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.924136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0fe7c-5412-4336-b552-867c52e35f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:15.925148 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0fe7c-5412-4336-b552-867c52e35f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:15.927145 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0fe7c-5412-4336-b552-867c52e35f2d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:15.946903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1142/2334] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:15.954658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286755-2942-4e7c-a151-29c4a6fa450e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:15.967985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:15.973185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286755-2942-4e7c-a151-29c4a6fa450e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:15.983059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:15.986608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:15.987291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:15.987291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] PATH_INFO: `/v3/projects/031e8090f49742eb9d7662cd74114622/users/92dca78daf8a435aa9214d2672750297/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:16.001272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=031e8090f49742eb9d7662cd74114622, user_id=92dca78daf8a435aa9214d2672750297, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:16.004637 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:16.030447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1193/2335] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:15 2022] PUT /identity/v3/projects/031e8090f49742eb9d7662cd74114622/users/92dca78daf8a435aa9214d2672750297/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:16.030680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db3d3-6615-432c-b767-873493bcff5b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:16.030900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db3d3-6615-432c-b767-873493bcff5b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:16.031182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db3d3-6615-432c-b767-873493bcff5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:16.061860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-df4db3d3-6615-432c-b767-873493bcff5b None None] MFA Rules not processed for user `92dca78daf8a435aa9214d2672750297`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:16.063110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-df4db3d3-6615-432c-b767-873493bcff5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:16.154472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1143/2336] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:16.168690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:16.172639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:16.286497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73e4c6-f742-411c-8d7a-c7513388f2dc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:16.291912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79488a-4053-47f5-87c5-6527504efead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:16.294523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79488a-4053-47f5-87c5-6527504efead service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:16.295058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79488a-4053-47f5-87c5-6527504efead service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:16.295510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79488a-4053-47f5-87c5-6527504efead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:16.389223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79488a-4053-47f5-87c5-6527504efead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:16.390877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79488a-4053-47f5-87c5-6527504efead service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:16.400983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1194/2337] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:17.765479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4db3d3-6615-432c-b767-873493bcff5b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:17.767641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-df4db3d3-6615-432c-b767-873493bcff5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:17.865666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4db3d3-6615-432c-b767-873493bcff5b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:17.873541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:17.874161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:17.877140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:17.877140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] PATH_INFO: `/v3/users/fe5a7da9323c4397bf44c3bd4d11bd28` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:17.878397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5a7da9323c4397bf44c3bd4d11bd28)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:17.884096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:17.954093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:17.969562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5a7da9323c4397bf44c3bd4d11bd28'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:17.987202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5a7da9323c4397bf44c3bd4d11bd28'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:17.987202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5a7da9323c4397bf44c3bd4d11bd28'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:18.008477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5a7da9323c4397bf44c3bd4d11bd28'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:18.016396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5a7da9323c4397bf44c3bd4d11bd28'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:18.044621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1144/2338] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:17 2022] DELETE /identity/v3/users/fe5a7da9323c4397bf44c3bd4d11bd28 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:18.049880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e6af-c054-4946-8385-952e27c850f4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:18.050383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e6af-c054-4946-8385-952e27c850f4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:18.051188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e6af-c054-4946-8385-952e27c850f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:18.088662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ed97e6af-c054-4946-8385-952e27c850f4 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:18.090514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ed97e6af-c054-4946-8385-952e27c850f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:18.217679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1195/2339] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:18.228296 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:18.232089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:18.326739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c393780-34c8-490c-b024-52158d16d386 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:18.331662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b06d7-b261-447b-b7e5-3b977fc15acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:18.332426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b06d7-b261-447b-b7e5-3b977fc15acd service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:18.332759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b06d7-b261-447b-b7e5-3b977fc15acd service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:18.333183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b06d7-b261-447b-b7e5-3b977fc15acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:18.357415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b06d7-b261-447b-b7e5-3b977fc15acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:18.359751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b06d7-b261-447b-b7e5-3b977fc15acd service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:18.387186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1145/2340] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:18.948596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97e6af-c054-4946-8385-952e27c850f4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:18.951394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ed97e6af-c054-4946-8385-952e27c850f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:18.968292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97e6af-c054-4946-8385-952e27c850f4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:18.986380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:18.987175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:18.988576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:18.989166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] PATH_INFO: `/v3/projects/e52da7e888bd498892ff4c4878b83da6` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.002884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52da7e888bd498892ff4c4878b83da6)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:19.002884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:19.048383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:19.055739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52da7e888bd498892ff4c4878b83da6', 'request_id': 'req-ca3072d7-4abe-4045-a19f-83586349771c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:19.068398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52da7e888bd498892ff4c4878b83da6', 'request_id': 'req-ca3072d7-4abe-4045-a19f-83586349771c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:19.087820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52da7e888bd498892ff4c4878b83da6', 'request_id': 'req-ca3072d7-4abe-4045-a19f-83586349771c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:19.103859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] The token cache is being invalidate because project e52da7e888bd498892ff4c4878b83da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:19.104204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:19.104917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ca3072d7-4abe-4045-a19f-83586349771c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:19.106997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1196/2341] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:18 2022] DELETE /identity/v3/projects/e52da7e888bd498892ff4c4878b83da6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:19.111799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc005d6-36e9-4505-8585-81314629d994 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.112281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc005d6-36e9-4505-8585-81314629d994 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.112419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc005d6-36e9-4505-8585-81314629d994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.132859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fcc005d6-36e9-4505-8585-81314629d994 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:19.133998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fcc005d6-36e9-4505-8585-81314629d994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:19.208927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1146/2342] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:19.216862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885342e-6034-467b-a619-112909805659 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.217156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885342e-6034-467b-a619-112909805659 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.217505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885342e-6034-467b-a619-112909805659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.255214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9885342e-6034-467b-a619-112909805659 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:19.256476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9885342e-6034-467b-a619-112909805659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:19.281325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1197/2343] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:19.281980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc005d6-36e9-4505-8585-81314629d994 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:19.293756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc005d6-36e9-4505-8585-81314629d994 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:19.303387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:19.304033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.304311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.304582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.305380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:39:19.306253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:19.307484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:19.321465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1147/2344] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:39:19.325640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885342e-6034-467b-a619-112909805659 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:19.336218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885342e-6034-467b-a619-112909805659 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:19.341180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f59195-7cb7-4286-a4ff-448fc1cbefe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:19.341839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f59195-7cb7-4286-a4ff-448fc1cbefe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.342121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f59195-7cb7-4286-a4ff-448fc1cbefe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.342396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f59195-7cb7-4286-a4ff-448fc1cbefe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.343077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f59195-7cb7-4286-a4ff-448fc1cbefe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:19.345228 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f59195-7cb7-4286-a4ff-448fc1cbefe3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:19.397004 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:19.404705 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1198/2345] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:19 2022] POST /identity/v3/projects => generated 381 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:19.408798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27e8f9-7e7c-4e4c-b270-2b6b1a5f8b6a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:19.469851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:19.493702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e93b33-c322-4da3-8c7f-85540abf827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:19.494635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93b33-c322-4da3-8c7f-85540abf827f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.495010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93b33-c322-4da3-8c7f-85540abf827f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.495506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93b33-c322-4da3-8c7f-85540abf827f admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.496308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e93b33-c322-4da3-8c7f-85540abf827f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:19.498478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e93b33-c322-4da3-8c7f-85540abf827f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:19.531853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f59195-7cb7-4286-a4ff-448fc1cbefe3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:19.542559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1148/2346] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:19 2022] POST /identity/v3/users => generated 291 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:19.543709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f59195-7cb7-4286-a4ff-448fc1cbefe3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:19.548593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb123bb-abb9-437a-ab55-4b2df177dffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:19.550474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb123bb-abb9-437a-ab55-4b2df177dffa admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.550852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb123bb-abb9-437a-ab55-4b2df177dffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.551523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb123bb-abb9-437a-ab55-4b2df177dffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.551812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb123bb-abb9-437a-ab55-4b2df177dffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:19.553513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb123bb-abb9-437a-ab55-4b2df177dffa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:19.566542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1199/2347] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:19.570777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e93b33-c322-4da3-8c7f-85540abf827f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:19.584226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e93b33-c322-4da3-8c7f-85540abf827f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:19.590880 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e399f-98f4-4eb2-8838-09866c6f24ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:19.591544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e399f-98f4-4eb2-8838-09866c6f24ed admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.591827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e399f-98f4-4eb2-8838-09866c6f24ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.592106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e399f-98f4-4eb2-8838-09866c6f24ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.592815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e399f-98f4-4eb2-8838-09866c6f24ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:19.595941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e399f-98f4-4eb2-8838-09866c6f24ed admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:19.613329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1149/2348] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:19.617562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb123bb-abb9-437a-ab55-4b2df177dffa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:19.628446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb123bb-abb9-437a-ab55-4b2df177dffa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:19.641461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:19.648269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.648720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.648837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] PATH_INFO: `/v3/projects/bafc15c173f94f8eb804e56a2d4673a8/users/bb9f183ac716439aab70e789f9561901/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.684167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafc15c173f94f8eb804e56a2d4673a8, user_id=bb9f183ac716439aab70e789f9561901, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:19.687079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:19.706268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1200/2349] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:19 2022] PUT /identity/v3/projects/bafc15c173f94f8eb804e56a2d4673a8/users/bb9f183ac716439aab70e789f9561901/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:39:19.712529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad2f2c-a45f-40a9-b42c-5fca20793d61 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.713403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad2f2c-a45f-40a9-b42c-5fca20793d61 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.713943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad2f2c-a45f-40a9-b42c-5fca20793d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.738780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-caad2f2c-a45f-40a9-b42c-5fca20793d61 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:19.740021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-caad2f2c-a45f-40a9-b42c-5fca20793d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:19.838370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1150/2350] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:19.852990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:19.856867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:19.953005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5a871-fe2e-426d-a9ed-d6a73ca53ada admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:19.957642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:19.958271 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:19.958553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:19.958828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:19.969849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:19.973854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:19.984719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1201/2351] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:20.538236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c113-ecba-4bc5-a99f-6ea9663eec0e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:20.539189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c113-ecba-4bc5-a99f-6ea9663eec0e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:20.539673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c113-ecba-4bc5-a99f-6ea9663eec0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:20.589855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-0830c113-ecba-4bc5-a99f-6ea9663eec0e None None] MFA Rules not processed for user `93dacbaabf7343bbae337761b8d8cd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:20.591111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0830c113-ecba-4bc5-a99f-6ea9663eec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:20.676341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1151/2352] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:20.681736 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:20.683289 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:20.698600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845f55b-2c3c-425b-ab80-4df39ed60b0f service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:20.704552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4e1ca-c55f-44eb-a59a-330dac818f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:20.705559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e1ca-c55f-44eb-a59a-330dac818f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:20.705984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e1ca-c55f-44eb-a59a-330dac818f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:20.706282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e1ca-c55f-44eb-a59a-330dac818f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:20.706969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4e1ca-c55f-44eb-a59a-330dac818f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:20.708915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4e1ca-c55f-44eb-a59a-330dac818f0a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:20.742826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1202/2353] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:20 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:20.747011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830c113-ecba-4bc5-a99f-6ea9663eec0e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:20.760261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830c113-ecba-4bc5-a99f-6ea9663eec0e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:20.820347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:20.835220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed5f3-d981-4fa9-8967-f4939cd110e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:20.836419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed5f3-d981-4fa9-8967-f4939cd110e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:20.837478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed5f3-d981-4fa9-8967-f4939cd110e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:20.837778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed5f3-d981-4fa9-8967-f4939cd110e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:20.838788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eed5f3-d981-4fa9-8967-f4939cd110e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:20.841159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eed5f3-d981-4fa9-8967-f4939cd110e3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:20.874230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1152/2354] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:20 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:20.876042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4e1ca-c55f-44eb-a59a-330dac818f0a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:20.888531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4e1ca-c55f-44eb-a59a-330dac818f0a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:20.894296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f138f1-2118-49df-a7a6-17792458489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:20.894922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f138f1-2118-49df-a7a6-17792458489b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:20.895207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f138f1-2118-49df-a7a6-17792458489b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:20.895479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f138f1-2118-49df-a7a6-17792458489b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:20.896185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f138f1-2118-49df-a7a6-17792458489b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:20.897811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f138f1-2118-49df-a7a6-17792458489b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:20.914749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1203/2355] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:20.915912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed5f3-d981-4fa9-8967-f4939cd110e3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:20.933761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed5f3-d981-4fa9-8967-f4939cd110e3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:20.940241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a4e0-77a7-48fd-9be4-9290b1e9fa93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:20.941080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a4e0-77a7-48fd-9be4-9290b1e9fa93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:20.941510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a4e0-77a7-48fd-9be4-9290b1e9fa93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:20.941901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a4e0-77a7-48fd-9be4-9290b1e9fa93 admin admin] PATH_INFO: `/v3/projects/e44ec68574a94e0ebc807f38d47b76cb/users/0bef5e8c46f2481eb433383cd8c6e0a5/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:20.961144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07a4e0-77a7-48fd-9be4-9290b1e9fa93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44ec68574a94e0ebc807f38d47b76cb, user_id=0bef5e8c46f2481eb433383cd8c6e0a5, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:20.964241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07a4e0-77a7-48fd-9be4-9290b1e9fa93 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:20.981287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1153/2356] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:20 2022] PUT /identity/v3/projects/e44ec68574a94e0ebc807f38d47b76cb/users/0bef5e8c46f2481eb433383cd8c6e0a5/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:20.985401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f138f1-2118-49df-a7a6-17792458489b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:20.996719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f138f1-2118-49df-a7a6-17792458489b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:21.047326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:21.083322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c43932-966d-41da-81fd-7e0ba992338c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:21.084117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c43932-966d-41da-81fd-7e0ba992338c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:21.084402 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c43932-966d-41da-81fd-7e0ba992338c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:21.084676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c43932-966d-41da-81fd-7e0ba992338c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:21.086053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c43932-966d-41da-81fd-7e0ba992338c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:21.087420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c43932-966d-41da-81fd-7e0ba992338c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:21.101535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1204/2357] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:39:21.105508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a4e0-77a7-48fd-9be4-9290b1e9fa93 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:21.117227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a4e0-77a7-48fd-9be4-9290b1e9fa93 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:21.122184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:21.122860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:21.123138 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:21.123411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] PATH_INFO: `/v3/projects/e44ec68574a94e0ebc807f38d47b76cb/users/0bef5e8c46f2481eb433383cd8c6e0a5/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:21.125535 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44ec68574a94e0ebc807f38d47b76cb, user_id=0bef5e8c46f2481eb433383cd8c6e0a5, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:21.128975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:21.140408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1154/2358] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:21 2022] PUT /identity/v3/projects/e44ec68574a94e0ebc807f38d47b76cb/users/0bef5e8c46f2481eb433383cd8c6e0a5/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:21.378049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f748f7-22ea-4f3a-b398-ca5d6fdb5c5f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:21.378276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f748f7-22ea-4f3a-b398-ca5d6fdb5c5f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:21.378531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f748f7-22ea-4f3a-b398-ca5d6fdb5c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:21.486785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d5f748f7-22ea-4f3a-b398-ca5d6fdb5c5f None None] MFA Rules not processed for user `bb9f183ac716439aab70e789f9561901`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:21.488582 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d5f748f7-22ea-4f3a-b398-ca5d6fdb5c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:21.654908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1205/2359] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:21.687414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:21.691082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:21.773248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f748f7-22ea-4f3a-b398-ca5d6fdb5c5f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:21.779768 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d5f748f7-22ea-4f3a-b398-ca5d6fdb5c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:21.793389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a0724-4def-4b47-b9bc-a212046a9420 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:21.799239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdf668-10fb-4871-be25-2b8f1d1284d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:21.800261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdf668-10fb-4871-be25-2b8f1d1284d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:21.800731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdf668-10fb-4871-be25-2b8f1d1284d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:21.801346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdf668-10fb-4871-be25-2b8f1d1284d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:21.811733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f748f7-22ea-4f3a-b398-ca5d6fdb5c5f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:21.815620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdf668-10fb-4871-be25-2b8f1d1284d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:21.817375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdf668-10fb-4871-be25-2b8f1d1284d2 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:21.826600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20661030-4689-4270-90a3-2bfec34be680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:21.828764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20661030-4689-4270-90a3-2bfec34be680 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:21.830605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20661030-4689-4270-90a3-2bfec34be680 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:21.830605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20661030-4689-4270-90a3-2bfec34be680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:21.831554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1155/2360] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:21.834522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-20661030-4689-4270-90a3-2bfec34be680 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:21.957575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20661030-4689-4270-90a3-2bfec34be680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:21.959021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20661030-4689-4270-90a3-2bfec34be680 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:21.973571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1206/2361] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:39:23.505976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36db3d-ab72-4ec9-ab79-ade3558ccbce None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:23.506645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36db3d-ab72-4ec9-ab79-ade3558ccbce None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:23.506645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36db3d-ab72-4ec9-ab79-ade3558ccbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:23.582156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1c36db3d-ab72-4ec9-ab79-ade3558ccbce None None] MFA Rules not processed for user `bb9f183ac716439aab70e789f9561901`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:23.583666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1c36db3d-ab72-4ec9-ab79-ade3558ccbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:23.633194 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1156/2362] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:28.036926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392c2f-a3d6-42d7-98a6-28c6c93f27f0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:28.037463 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392c2f-a3d6-42d7-98a6-28c6c93f27f0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:28.037563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392c2f-a3d6-42d7-98a6-28c6c93f27f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:28.088975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-05392c2f-a3d6-42d7-98a6-28c6c93f27f0 None None] MFA Rules not processed for user `0bef5e8c46f2481eb433383cd8c6e0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:28.090881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-05392c2f-a3d6-42d7-98a6-28c6c93f27f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:28.145601 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:28.197375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1207/2363] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:28.204110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36db3d-ab72-4ec9-ab79-ade3558ccbce None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:28.208281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1c36db3d-ab72-4ec9-ab79-ade3558ccbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:28.307991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36db3d-ab72-4ec9-ab79-ade3558ccbce None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:28.315689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '649334bb7ed04bd69eb971fc070d96e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:28.316709 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:28.317224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:28.317652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:28.401827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:28.404960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:28.406747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:28.419381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1157/2364] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:28.697133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5f895-cbb3-43fc-8bab-07654ef2fe5f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:28.699262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5f895-cbb3-43fc-8bab-07654ef2fe5f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:28.703295 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5f895-cbb3-43fc-8bab-07654ef2fe5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:28.728695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2df5f895-cbb3-43fc-8bab-07654ef2fe5f None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:28.729963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2df5f895-cbb3-43fc-8bab-07654ef2fe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:28.868296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1208/2365] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:39:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:28.933432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:28.935677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:29.076292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeae257-211f-4b94-b37a-bee7125dbe6c service glance] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:29.087486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:29.088476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:29.089371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:29.089924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:29.106235 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:29.108520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:29.121079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1158/2366] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:39:29.156900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076609c-055b-4487-aa28-0be33ae16a88 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:29.157392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076609c-055b-4487-aa28-0be33ae16a88 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:29.158182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076609c-055b-4487-aa28-0be33ae16a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:29.187813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0076609c-055b-4487-aa28-0be33ae16a88 None None] MFA Rules not processed for user `ffc6812703254c51a4200abf3dee2d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:29.189419 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0076609c-055b-4487-aa28-0be33ae16a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:29.289479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1209/2367] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:29.857820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:29.866376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:30.046364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f033e-664e-46d3-b5c7-0bd505f910ee service swift] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:30.053135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:30.054050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:30.054333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:30.054609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] PATH_INFO: `/v3/users/7f9f6b845bd04e8bb50d7c554c004995` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:30.066272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f9f6b845bd04e8bb50d7c554c004995)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:30.068253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:30.153822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f9f6b845bd04e8bb50d7c554c004995'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:30.165347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f9f6b845bd04e8bb50d7c554c004995'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:30.166280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f9f6b845bd04e8bb50d7c554c004995'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:30.189800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f9f6b845bd04e8bb50d7c554c004995'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:30.192404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f9f6b845bd04e8bb50d7c554c004995'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:30.222355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1159/2368] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:29 2022] DELETE /identity/v3/users/7f9f6b845bd04e8bb50d7c554c004995 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:30.237464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a889a99-185d-4fee-a055-8279298ea48b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:30.237932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a889a99-185d-4fee-a055-8279298ea48b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:30.238287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a889a99-185d-4fee-a055-8279298ea48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:30.275218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1a889a99-185d-4fee-a055-8279298ea48b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:30.278629 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1a889a99-185d-4fee-a055-8279298ea48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:30.490100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1210/2369] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:30.500441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:30.502925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:30.520096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:30.623949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed3a2a-c767-484b-8fd0-6156a1ca0b08 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:30.636458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c089ad9-23e1-46b1-b326-b543b5cdb2a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:30.639408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c089ad9-23e1-46b1-b326-b543b5cdb2a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:30.640845 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c089ad9-23e1-46b1-b326-b543b5cdb2a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:30.642998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c089ad9-23e1-46b1-b326-b543b5cdb2a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:30.680408 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c089ad9-23e1-46b1-b326-b543b5cdb2a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:30.683273 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c089ad9-23e1-46b1-b326-b543b5cdb2a5 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:30.714029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1160/2370] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:31.744183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a889a99-185d-4fee-a055-8279298ea48b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:31.758006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1a889a99-185d-4fee-a055-8279298ea48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:31.772474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a889a99-185d-4fee-a055-8279298ea48b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:31.784084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:31.784835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:31.785193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:31.785576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] PATH_INFO: `/v3/projects/c44e432e64504eefb5f8ae47bb679270` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:31.789050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c44e432e64504eefb5f8ae47bb679270)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:31.791428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:31.840597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:31.865228 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c44e432e64504eefb5f8ae47bb679270', 'request_id': 'req-9998cfd8-71ba-4035-8b45-b28af0d33626'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:31.885504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c44e432e64504eefb5f8ae47bb679270', 'request_id': 'req-9998cfd8-71ba-4035-8b45-b28af0d33626'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:31.894417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c44e432e64504eefb5f8ae47bb679270', 'request_id': 'req-9998cfd8-71ba-4035-8b45-b28af0d33626'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:31.956395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] The token cache is being invalidate because project c44e432e64504eefb5f8ae47bb679270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:31.978327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:31.979237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:31.983631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1211/2371] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:31 2022] DELETE /identity/v3/projects/c44e432e64504eefb5f8ae47bb679270 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:31.985307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c089ad9-23e1-46b1-b326-b543b5cdb2a5 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:32.008288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c089ad9-23e1-46b1-b326-b543b5cdb2a5 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:32.090028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:32.091424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:32.091820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:32.092209 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] PATH_INFO: `/v3/users/92dca78daf8a435aa9214d2672750297` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:32.093575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92dca78daf8a435aa9214d2672750297)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:32.096898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:32.180518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92dca78daf8a435aa9214d2672750297'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:32.195730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92dca78daf8a435aa9214d2672750297'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:32.196714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92dca78daf8a435aa9214d2672750297'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:32.217368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92dca78daf8a435aa9214d2672750297'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:32.218033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f0e250d4-8fe7-4304-8761-7d403f5bf6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92dca78daf8a435aa9214d2672750297'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:32.237480 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1161/2372] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:39:31 2022] DELETE /identity/v3/users/92dca78daf8a435aa9214d2672750297 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:32.800873 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:32.803014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:32.817877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998cfd8-71ba-4035-8b45-b28af0d33626 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:32.955131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:32.956373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:32.956900 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:32.957420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] PATH_INFO: `/v3/projects/031e8090f49742eb9d7662cd74114622` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:32.958971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031e8090f49742eb9d7662cd74114622)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:32.961097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:33.045321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031e8090f49742eb9d7662cd74114622', 'request_id': 'req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:33.055877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031e8090f49742eb9d7662cd74114622', 'request_id': 'req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:33.065370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031e8090f49742eb9d7662cd74114622', 'request_id': 'req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:33.090471 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] The token cache is being invalidate because project 031e8090f49742eb9d7662cd74114622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:39:33.092722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:33.093891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1cd96ed1-4137-4a4b-8b88-02a9a43f9292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:39:33.096506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1212/2373] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:39:32 2022] DELETE /identity/v3/projects/031e8090f49742eb9d7662cd74114622 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:33.102292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4c0ce-55d7-4e30-b302-f39f701124aa None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:33.102852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4c0ce-55d7-4e30-b302-f39f701124aa None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:33.102852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4c0ce-55d7-4e30-b302-f39f701124aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:33.173803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-67f4c0ce-55d7-4e30-b302-f39f701124aa None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:33.179902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-67f4c0ce-55d7-4e30-b302-f39f701124aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:33.245227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:33.346028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1162/2374] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:33.346203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026be165-08e9-4c2a-9a6d-4819e7a50242 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:33.346750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026be165-08e9-4c2a-9a6d-4819e7a50242 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:33.347110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026be165-08e9-4c2a-9a6d-4819e7a50242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:33.417056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-026be165-08e9-4c2a-9a6d-4819e7a50242 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:33.418805 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-026be165-08e9-4c2a-9a6d-4819e7a50242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:33.475850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1213/2375] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:33.480090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4c0ce-55d7-4e30-b302-f39f701124aa None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:33.491788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4c0ce-55d7-4e30-b302-f39f701124aa None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:33.507359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:33.508617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:33.509121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:33.515308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:33.516087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:39:33.516398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:33.519056 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:33.536111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1163/2376] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:39:33.537389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026be165-08e9-4c2a-9a6d-4819e7a50242 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:33.561880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026be165-08e9-4c2a-9a6d-4819e7a50242 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:33.577244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f734d-62e2-4646-86a3-2b376360de22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:33.578619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f734d-62e2-4646-86a3-2b376360de22 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:33.579120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f734d-62e2-4646-86a3-2b376360de22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:33.579751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f734d-62e2-4646-86a3-2b376360de22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:33.580673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f734d-62e2-4646-86a3-2b376360de22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:33.583669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f734d-62e2-4646-86a3-2b376360de22 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:33.651004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1214/2377] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:39:33 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:39:33.651532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:33.669497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd01c-99c4-4fa3-8b79-610676a84a67 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:33.712475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49398c14-2893-4d27-8994-22fb5d190140 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:33.713243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49398c14-2893-4d27-8994-22fb5d190140 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:33.713652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49398c14-2893-4d27-8994-22fb5d190140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:33.742193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-49398c14-2893-4d27-8994-22fb5d190140 None None] MFA Rules not processed for user `13ca4be686f84c279f87016c20129833`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:33.743778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-49398c14-2893-4d27-8994-22fb5d190140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:33.777619 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:33.832672 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08abde-58e8-4f70-92c6-3ea00d7ed317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:33.833648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08abde-58e8-4f70-92c6-3ea00d7ed317 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:33.834415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08abde-58e8-4f70-92c6-3ea00d7ed317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:33.834882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08abde-58e8-4f70-92c6-3ea00d7ed317 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:33.836188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08abde-58e8-4f70-92c6-3ea00d7ed317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:33.840932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08abde-58e8-4f70-92c6-3ea00d7ed317 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:33.883811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1215/2378] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:33.916594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1164/2379] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:39:33 2022] POST /identity/v3/users => generated 286 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:39:33.924294 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49398c14-2893-4d27-8994-22fb5d190140 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:33.979523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49398c14-2893-4d27-8994-22fb5d190140 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:33.992342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee602ec-e82d-4c0c-9150-5d5a618f0791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:33.993144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee602ec-e82d-4c0c-9150-5d5a618f0791 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:33.993595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee602ec-e82d-4c0c-9150-5d5a618f0791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:33.993963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee602ec-e82d-4c0c-9150-5d5a618f0791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:33.997022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee602ec-e82d-4c0c-9150-5d5a618f0791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:33.997821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee602ec-e82d-4c0c-9150-5d5a618f0791 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:34.028299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1216/2380] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:34.029858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08abde-58e8-4f70-92c6-3ea00d7ed317 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:34.051414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08abde-58e8-4f70-92c6-3ea00d7ed317 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:34.058086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668f936-c35e-44aa-9d66-e2841f3de0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:34.059641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f936-c35e-44aa-9d66-e2841f3de0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:34.060108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f936-c35e-44aa-9d66-e2841f3de0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:34.061222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f936-c35e-44aa-9d66-e2841f3de0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:34.062326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668f936-c35e-44aa-9d66-e2841f3de0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:34.064775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668f936-c35e-44aa-9d66-e2841f3de0b1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:34.093926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1165/2381] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:39:34.096050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee602ec-e82d-4c0c-9150-5d5a618f0791 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:34.117337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee602ec-e82d-4c0c-9150-5d5a618f0791 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:34.142029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:34.143654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:34.143654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:34.143654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] PATH_INFO: `/v3/projects/e6b094da947d4a4783d12cd0755c0f2b/users/5ea0d414f442428ab2cbaab378fcf70c/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:34.189654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b094da947d4a4783d12cd0755c0f2b, user_id=5ea0d414f442428ab2cbaab378fcf70c, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:34.195954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:34.232989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1217/2382] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:39:34 2022] PUT /identity/v3/projects/e6b094da947d4a4783d12cd0755c0f2b/users/5ea0d414f442428ab2cbaab378fcf70c/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:39:34.244665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb9f5-8c04-4839-b802-c7cd7c5ad970 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:34.245249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb9f5-8c04-4839-b802-c7cd7c5ad970 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:34.245449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb9f5-8c04-4839-b802-c7cd7c5ad970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:34.377599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-924eb9f5-8c04-4839-b802-c7cd7c5ad970 None None] MFA Rules not processed for user `5ea0d414f442428ab2cbaab378fcf70c`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:34.383159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-924eb9f5-8c04-4839-b802-c7cd7c5ad970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:34.522421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1166/2383] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:34.535041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:34.536869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:34.630520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490d489-b533-4491-a563-5d3bacfeab94 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:34.641183 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e871700-bd21-448e-8f40-0de4062dc75d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '649334bb7ed04bd69eb971fc070d96e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:34.643051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871700-bd21-448e-8f40-0de4062dc75d service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:34.643500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871700-bd21-448e-8f40-0de4062dc75d service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:34.643940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871700-bd21-448e-8f40-0de4062dc75d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:34.660569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e871700-bd21-448e-8f40-0de4062dc75d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:34.662643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e871700-bd21-448e-8f40-0de4062dc75d service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:34.674764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1218/2384] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:39:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:39:44.278801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa7aea-f0d7-4384-bddc-e63a56c3e3d5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:44.279396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa7aea-f0d7-4384-bddc-e63a56c3e3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:44.279396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa7aea-f0d7-4384-bddc-e63a56c3e3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:44.300385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e1fa7aea-f0d7-4384-bddc-e63a56c3e3d5 None None] MFA Rules not processed for user `93dacbaabf7343bbae337761b8d8cd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:44.301639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e1fa7aea-f0d7-4384-bddc-e63a56c3e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:44.319265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:44.362188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1167/2385] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:44.668267 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:53.945864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a714-b0e6-45c3-a7f6-6f2c0eed7b35 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:53.946851 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a714-b0e6-45c3-a7f6-6f2c0eed7b35 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:53.947158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a714-b0e6-45c3-a7f6-6f2c0eed7b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:53.966719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cca5a714-b0e6-45c3-a7f6-6f2c0eed7b35 None None] MFA Rules not processed for user `4a1db92e28444854b9de817dd36104ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:53.967923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cca5a714-b0e6-45c3-a7f6-6f2c0eed7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:54.032435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1219/2386] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:39:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:39:54.353118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=15 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:39:56.157240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a998c6-14a0-4424-b10b-d8c9efab97f0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:56.157970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a998c6-14a0-4424-b10b-d8c9efab97f0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:56.157970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a998c6-14a0-4424-b10b-d8c9efab97f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:56.183545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a5a998c6-14a0-4424-b10b-d8c9efab97f0 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:39:56.185280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a5a998c6-14a0-4424-b10b-d8c9efab97f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:56.260639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1168/2387] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:39:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:39:56.282530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5a714-b0e6-45c3-a7f6-6f2c0eed7b35 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:39:56.285872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cca5a714-b0e6-45c3-a7f6-6f2c0eed7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:39:56.400700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5a714-b0e6-45c3-a7f6-6f2c0eed7b35 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:39:56.407044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b4d83-7c19-4b02-a1d5-9afd48a9fa7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:39:56.409494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b4d83-7c19-4b02-a1d5-9afd48a9fa7c service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:39:56.409494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b4d83-7c19-4b02-a1d5-9afd48a9fa7c service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:39:56.409494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b4d83-7c19-4b02-a1d5-9afd48a9fa7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:39:56.445117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b4d83-7c19-4b02-a1d5-9afd48a9fa7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:39:56.452339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b4d83-7c19-4b02-a1d5-9afd48a9fa7c service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:39:56.488567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1220/2388] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:39:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:40:06.250543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=25 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:06.485345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=57 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:12.429350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a998c6-14a0-4424-b10b-d8c9efab97f0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:12.429350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a5a998c6-14a0-4424-b10b-d8c9efab97f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:12.527772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a998c6-14a0-4424-b10b-d8c9efab97f0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:12.534576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf134e7-342f-402a-aa76-ccb464d9367b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:12.535238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf134e7-342f-402a-aa76-ccb464d9367b service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:12.535524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf134e7-342f-402a-aa76-ccb464d9367b service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:12.535802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf134e7-342f-402a-aa76-ccb464d9367b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:12.537697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-caf134e7-342f-402a-aa76-ccb464d9367b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:12.550795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf134e7-342f-402a-aa76-ccb464d9367b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:12.552392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf134e7-342f-402a-aa76-ccb464d9367b service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:12.563646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1169/2389] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:17.998217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71449a-8de3-4dc8-9808-236b4444c912 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:17.998732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71449a-8de3-4dc8-9808-236b4444c912 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:17.998844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71449a-8de3-4dc8-9808-236b4444c912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:18.018817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9f71449a-8de3-4dc8-9808-236b4444c912 None None] MFA Rules not processed for user `a2080efc6cbd471995593012bc268765`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:18.020000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9f71449a-8de3-4dc8-9808-236b4444c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:18.075707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1221/2390] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:40:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:19.961770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5f370-47ab-4633-875c-f956fdf1d8e6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:19.962606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5f370-47ab-4633-875c-f956fdf1d8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:19.963094 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5f370-47ab-4633-875c-f956fdf1d8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:20.006696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6da5f370-47ab-4633-875c-f956fdf1d8e6 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:20.007898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6da5f370-47ab-4633-875c-f956fdf1d8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:20.035406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1170/2391] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:40:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:20.049256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71449a-8de3-4dc8-9808-236b4444c912 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:20.061927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71449a-8de3-4dc8-9808-236b4444c912 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:20.067522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca856b-27c4-4e49-9abc-b90a82212fe5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:20.068149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca856b-27c4-4e49-9abc-b90a82212fe5 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:20.068725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca856b-27c4-4e49-9abc-b90a82212fe5 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:20.069010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca856b-27c4-4e49-9abc-b90a82212fe5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:20.078393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca856b-27c4-4e49-9abc-b90a82212fe5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:20.080099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca856b-27c4-4e49-9abc-b90a82212fe5 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:20.093491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1222/2392] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:40:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:40:20.893583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5f370-47ab-4633-875c-f956fdf1d8e6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:20.895485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6da5f370-47ab-4633-875c-f956fdf1d8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:20.977610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5f370-47ab-4633-875c-f956fdf1d8e6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:20.988598 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c1da4-76e8-4145-ae06-d602820eb67d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:20.989172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c1da4-76e8-4145-ae06-d602820eb67d service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:20.989494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c1da4-76e8-4145-ae06-d602820eb67d service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:20.989778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c1da4-76e8-4145-ae06-d602820eb67d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:20.997457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-939c1da4-76e8-4145-ae06-d602820eb67d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:21.073872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca856b-27c4-4e49-9abc-b90a82212fe5 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:21.258510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca856b-27c4-4e49-9abc-b90a82212fe5 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:21.264670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c1da4-76e8-4145-ae06-d602820eb67d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:21.273567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:21.275759 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c1da4-76e8-4145-ae06-d602820eb67d service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:21.276018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:21.276482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:21.277434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] PATH_INFO: `/v3/users/5ea0d414f442428ab2cbaab378fcf70c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:21.282119 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea0d414f442428ab2cbaab378fcf70c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:21.288189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:21.304576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1171/2393] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:21.427773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea0d414f442428ab2cbaab378fcf70c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:21.449349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea0d414f442428ab2cbaab378fcf70c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:21.463345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea0d414f442428ab2cbaab378fcf70c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:21.469351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea0d414f442428ab2cbaab378fcf70c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:21.502989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea0d414f442428ab2cbaab378fcf70c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:21.520363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1223/2394] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:40:21 2022] DELETE /identity/v3/users/5ea0d414f442428ab2cbaab378fcf70c => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:40:21.525152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377149b-4430-4797-a231-4a6108658e8a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:21.525667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377149b-4430-4797-a231-4a6108658e8a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:21.526052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377149b-4430-4797-a231-4a6108658e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:21.556409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4377149b-4430-4797-a231-4a6108658e8a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:21.557836 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4377149b-4430-4797-a231-4a6108658e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:21.612856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:21.739281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1172/2395] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:21.752859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:21.754954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:21.778230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:21.917551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eca07-1666-4152-8cff-6eb20bda4044 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:21.930169 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae25cf1-dd36-447d-b221-710917300593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:21.930819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae25cf1-dd36-447d-b221-710917300593 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:21.931101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae25cf1-dd36-447d-b221-710917300593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:21.931377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae25cf1-dd36-447d-b221-710917300593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:21.969460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae25cf1-dd36-447d-b221-710917300593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:21.972899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae25cf1-dd36-447d-b221-710917300593 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:21.991473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1224/2396] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:23.792939 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377149b-4430-4797-a231-4a6108658e8a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:23.794714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4377149b-4430-4797-a231-4a6108658e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:23.834683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377149b-4430-4797-a231-4a6108658e8a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:23.852730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:23.853474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:23.853860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:23.854241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] PATH_INFO: `/v3/projects/e6b094da947d4a4783d12cd0755c0f2b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:23.858617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b094da947d4a4783d12cd0755c0f2b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:23.861338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:23.966805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b094da947d4a4783d12cd0755c0f2b', 'request_id': 'req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:23.977994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b094da947d4a4783d12cd0755c0f2b', 'request_id': 'req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:23.985335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b094da947d4a4783d12cd0755c0f2b', 'request_id': 'req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:24.005941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] The token cache is being invalidate because project e6b094da947d4a4783d12cd0755c0f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:40:24.006505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:24.007722 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:24.010027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1173/2397] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:40:23 2022] DELETE /identity/v3/projects/e6b094da947d4a4783d12cd0755c0f2b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:28.179770 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef47baa-48e4-4533-bdc8-6a98be605e51 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:28.180346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef47baa-48e4-4533-bdc8-6a98be605e51 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:28.180346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef47baa-48e4-4533-bdc8-6a98be605e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:28.214668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-eef47baa-48e4-4533-bdc8-6a98be605e51 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:28.215922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-eef47baa-48e4-4533-bdc8-6a98be605e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:28.312532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1225/2398] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:40:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:28.333036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:28.334702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:28.432515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f868c-5d3c-4a1b-945a-0a83f605d94e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:28.477249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d884291-b6cd-434e-b675-644132e35e09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:28.482297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d884291-b6cd-434e-b675-644132e35e09 service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:28.483070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d884291-b6cd-434e-b675-644132e35e09 service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:28.483569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d884291-b6cd-434e-b675-644132e35e09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:28.518224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d884291-b6cd-434e-b675-644132e35e09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:28.520113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d884291-b6cd-434e-b675-644132e35e09 service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:28.535354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1174/2399] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:40:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:40:38.291365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=46 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:38.528910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:52.408923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e2f1c-95a2-4c88-9d69-765971c6c03e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:52.410817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e2f1c-95a2-4c88-9d69-765971c6c03e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:52.411161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e2f1c-95a2-4c88-9d69-765971c6c03e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:52.412871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1226/2400] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:40:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:40:52.426109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747f63e-795c-462e-a829-809c4cafe0c0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:52.427177 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747f63e-795c-462e-a829-809c4cafe0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:52.427683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747f63e-795c-462e-a829-809c4cafe0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:52.460375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7747f63e-795c-462e-a829-809c4cafe0c0 None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:52.461890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7747f63e-795c-462e-a829-809c4cafe0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:52.577498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1175/2401] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:40:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:52.589893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e2f1c-95a2-4c88-9d69-765971c6c03e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:52.591734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5e9e2f1c-95a2-4c88-9d69-765971c6c03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:52.674832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e2f1c-95a2-4c88-9d69-765971c6c03e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:52.680791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9efdb-2282-43c9-86a9-33c52ef62170 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '649334bb7ed04bd69eb971fc070d96e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:52.681661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9efdb-2282-43c9-86a9-33c52ef62170 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:52.682107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9efdb-2282-43c9-86a9-33c52ef62170 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:52.682547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9efdb-2282-43c9-86a9-33c52ef62170 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:52.696241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9efdb-2282-43c9-86a9-33c52ef62170 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:52.697917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9efdb-2282-43c9-86a9-33c52ef62170 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:52.711571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1227/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:52.857022 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcc3d-7615-4ee5-81d2-8b8aa86840ef None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:52.857229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcc3d-7615-4ee5-81d2-8b8aa86840ef None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:52.857534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcc3d-7615-4ee5-81d2-8b8aa86840ef None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:52.860908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1176/2403] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:40:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:40:52.871710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c168ba-6a64-4e66-8b3b-cddd8c29ed9b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:52.872051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c168ba-6a64-4e66-8b3b-cddd8c29ed9b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:52.872382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c168ba-6a64-4e66-8b3b-cddd8c29ed9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:52.924480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e4c168ba-6a64-4e66-8b3b-cddd8c29ed9b None None] MFA Rules not processed for user `4b46a15d481f4393bef8c4fa2281710b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:52.926182 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e4c168ba-6a64-4e66-8b3b-cddd8c29ed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:53.023990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1228/2404] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:40:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:40:53.041615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfcc3d-7615-4ee5-81d2-8b8aa86840ef None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:53.044352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-77dfcc3d-7615-4ee5-81d2-8b8aa86840ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:53.091846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfcc3d-7615-4ee5-81d2-8b8aa86840ef None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:53.101478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4734c-f863-46c2-bcd8-b2c3a3dd6ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:53.102793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4734c-f863-46c2-bcd8-b2c3a3dd6ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:53.103307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4734c-f863-46c2-bcd8-b2c3a3dd6ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:53.103592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4734c-f863-46c2-bcd8-b2c3a3dd6ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:53.112530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4734c-f863-46c2-bcd8-b2c3a3dd6ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:53.114041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4734c-f863-46c2-bcd8-b2c3a3dd6ab8 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:53.123309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1177/2405] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:53.223858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4484-99b8-47cd-af96-6e2165871b87 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:53.224240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4484-99b8-47cd-af96-6e2165871b87 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:53.224547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4484-99b8-47cd-af96-6e2165871b87 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:53.225995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1229/2406] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:40:53.231592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8f134-8e14-444b-b0e1-f2f7f20273bc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:53.232031 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8f134-8e14-444b-b0e1-f2f7f20273bc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:53.232318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8f134-8e14-444b-b0e1-f2f7f20273bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:53.255108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3bc8f134-8e14-444b-b0e1-f2f7f20273bc None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:53.256630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3bc8f134-8e14-444b-b0e1-f2f7f20273bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:53.268658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1178/2407] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:40:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:53.279587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f4484-99b8-47cd-af96-6e2165871b87 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:53.291243 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f4484-99b8-47cd-af96-6e2165871b87 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:53.294901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a514f-89f0-4ee2-be3d-2554c983503b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:53.295513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a514f-89f0-4ee2-be3d-2554c983503b service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:53.295789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a514f-89f0-4ee2-be3d-2554c983503b service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:53.296056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a514f-89f0-4ee2-be3d-2554c983503b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:53.308035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a514f-89f0-4ee2-be3d-2554c983503b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:53.309689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a514f-89f0-4ee2-be3d-2554c983503b service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:53.321418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1230/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:53.936871 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6ba40-5b23-49b3-8d14-15f66406e5e2 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:53.937588 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6ba40-5b23-49b3-8d14-15f66406e5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:53.938084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6ba40-5b23-49b3-8d14-15f66406e5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:53.957998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4ba6ba40-5b23-49b3-8d14-15f66406e5e2 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:53.959196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4ba6ba40-5b23-49b3-8d14-15f66406e5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:53.966738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1179/2409] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jul 28 06:40:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:40:53.979784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a514f-89f0-4ee2-be3d-2554c983503b service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:53.991533 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a514f-89f0-4ee2-be3d-2554c983503b service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:53.996602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c69e-4caf-4288-bef2-c31f58cc9006 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:53.997222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c69e-4caf-4288-bef2-c31f58cc9006 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:53.997539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c69e-4caf-4288-bef2-c31f58cc9006 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:53.997816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c69e-4caf-4288-bef2-c31f58cc9006 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:54.006632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4c69e-4caf-4288-bef2-c31f58cc9006 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:54.008197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4c69e-4caf-4288-bef2-c31f58cc9006 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:54.016434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1231/2410] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:40:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:40:56.136310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6ba40-5b23-49b3-8d14-15f66406e5e2 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.139898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4ba6ba40-5b23-49b3-8d14-15f66406e5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.215487 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6ba40-5b23-49b3-8d14-15f66406e5e2 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.221422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.222081 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.222363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.222637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] PATH_INFO: `/v3/users/ffc6812703254c51a4200abf3dee2d95` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.223828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc6812703254c51a4200abf3dee2d95)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:56.225784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:56.277328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc6812703254c51a4200abf3dee2d95'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.286035 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc6812703254c51a4200abf3dee2d95'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.286677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc6812703254c51a4200abf3dee2d95'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.298481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:56.300886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc6812703254c51a4200abf3dee2d95'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.301855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fdf8183f-d17b-4501-8ac1-dd8a4fb88b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc6812703254c51a4200abf3dee2d95'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.316531 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1180/2411] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:40:56 2022] DELETE /identity/v3/users/ffc6812703254c51a4200abf3dee2d95 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:56.511536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c69e-4caf-4288-bef2-c31f58cc9006 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.525172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c69e-4caf-4288-bef2-c31f58cc9006 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.597109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.597869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.598149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.598421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] PATH_INFO: `/v3/projects/57601f159e244aa68924602c2c5dc385` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.599672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=57601f159e244aa68924602c2c5dc385)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:56.601646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:56.637365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:56.656155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57601f159e244aa68924602c2c5dc385', 'request_id': 'req-a8fbfffc-6f29-414d-9da0-807c9fd48bac'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.665578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57601f159e244aa68924602c2c5dc385', 'request_id': 'req-a8fbfffc-6f29-414d-9da0-807c9fd48bac'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.675112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57601f159e244aa68924602c2c5dc385', 'request_id': 'req-a8fbfffc-6f29-414d-9da0-807c9fd48bac'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.689571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] The token cache is being invalidate because project 57601f159e244aa68924602c2c5dc385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:40:56.689871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.690545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8fbfffc-6f29-414d-9da0-807c9fd48bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:56.692265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1232/2412] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:40:56 2022] DELETE /identity/v3/projects/57601f159e244aa68924602c2c5dc385 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:56.697670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02678f0a-8767-4d45-b926-b31a6101afe1 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.697966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02678f0a-8767-4d45-b926-b31a6101afe1 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.698476 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02678f0a-8767-4d45-b926-b31a6101afe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.721596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-02678f0a-8767-4d45-b926-b31a6101afe1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:56.722758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-02678f0a-8767-4d45-b926-b31a6101afe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.785339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1181/2413] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:56.793973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1e10-11bd-4241-97d7-3b817169dfe5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.794265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1e10-11bd-4241-97d7-3b817169dfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.794544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1e10-11bd-4241-97d7-3b817169dfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.815439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9aaf1e10-11bd-4241-97d7-3b817169dfe5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:56.816650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9aaf1e10-11bd-4241-97d7-3b817169dfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:56.826502 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1233/2414] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:56.831307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02678f0a-8767-4d45-b926-b31a6101afe1 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.841351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02678f0a-8767-4d45-b926-b31a6101afe1 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.846595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.847211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.847503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.847780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.848486 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:40:56.848795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:56.850514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:56.863026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1182/2415] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:40:56.867317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1e10-11bd-4241-97d7-3b817169dfe5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.877030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1e10-11bd-4241-97d7-3b817169dfe5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.881830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aaeaa-ed29-45db-b9d1-c86b6f7d9576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.882447 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aaeaa-ed29-45db-b9d1-c86b6f7d9576 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.882729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aaeaa-ed29-45db-b9d1-c86b6f7d9576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.883006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aaeaa-ed29-45db-b9d1-c86b6f7d9576 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.883963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aaeaa-ed29-45db-b9d1-c86b6f7d9576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:56.885893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aaeaa-ed29-45db-b9d1-c86b6f7d9576 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:56.914340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1234/2416] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:40:56 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:40:56.918488 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:56.930895 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb524d0-bfc9-46b7-a142-d0c63fe3908d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:56.996694 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb243b4-a3f6-4c70-8a11-d34906f31702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:56.997349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb243b4-a3f6-4c70-8a11-d34906f31702 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:56.997775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb243b4-a3f6-4c70-8a11-d34906f31702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:56.998070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb243b4-a3f6-4c70-8a11-d34906f31702 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:56.998753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb243b4-a3f6-4c70-8a11-d34906f31702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:57.001796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb243b4-a3f6-4c70-8a11-d34906f31702 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:57.030229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1183/2417] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:40:56 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:40:57.034382 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aaeaa-ed29-45db-b9d1-c86b6f7d9576 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:57.046796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aaeaa-ed29-45db-b9d1-c86b6f7d9576 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:57.052597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838138a-b694-4b93-92bc-63ca6c9d927f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:57.054102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838138a-b694-4b93-92bc-63ca6c9d927f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:57.054474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838138a-b694-4b93-92bc-63ca6c9d927f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:57.054884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838138a-b694-4b93-92bc-63ca6c9d927f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:57.058050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838138a-b694-4b93-92bc-63ca6c9d927f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:57.058050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838138a-b694-4b93-92bc-63ca6c9d927f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:57.074311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1235/2418] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:40:57.077913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb243b4-a3f6-4c70-8a11-d34906f31702 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:57.095807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb243b4-a3f6-4c70-8a11-d34906f31702 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:57.107586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb7b4a-a8c4-4ed6-b7e9-ed924c3b2583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:57.107780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7b4a-a8c4-4ed6-b7e9-ed924c3b2583 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:57.108039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7b4a-a8c4-4ed6-b7e9-ed924c3b2583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:57.108315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7b4a-a8c4-4ed6-b7e9-ed924c3b2583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:57.109024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb7b4a-a8c4-4ed6-b7e9-ed924c3b2583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:57.111442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb7b4a-a8c4-4ed6-b7e9-ed924c3b2583 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:57.123602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1184/2419] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:40:57.127729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838138a-b694-4b93-92bc-63ca6c9d927f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:57.138157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838138a-b694-4b93-92bc-63ca6c9d927f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:57.144673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:57.145410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:57.145990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:57.146285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] PATH_INFO: `/v3/projects/1643e2f438c54025968e23a4f8fbd7e2/users/b0d95431c0954828ba83fe5325e31f25/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:57.171710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1643e2f438c54025968e23a4f8fbd7e2, user_id=b0d95431c0954828ba83fe5325e31f25, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:57.175949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:57.213134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1236/2420] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:40:57 2022] PUT /identity/v3/projects/1643e2f438c54025968e23a4f8fbd7e2/users/b0d95431c0954828ba83fe5325e31f25/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:57.215922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febaa50c-5a70-419d-a362-4e3bda17b272 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:57.216415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febaa50c-5a70-419d-a362-4e3bda17b272 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:57.216656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febaa50c-5a70-419d-a362-4e3bda17b272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:57.267333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-febaa50c-5a70-419d-a362-4e3bda17b272 None None] MFA Rules not processed for user `b0d95431c0954828ba83fe5325e31f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:57.268691 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-febaa50c-5a70-419d-a362-4e3bda17b272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:57.319572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:57.376960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1185/2421] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:40:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:57.390324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:57.393829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:57.503496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68df87c-6db0-49fa-a0e3-c79225d79359 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:57.508195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc61e69-9142-4aab-95b4-c66db110ad09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:57.509132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61e69-9142-4aab-95b4-c66db110ad09 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:57.509658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61e69-9142-4aab-95b4-c66db110ad09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:57.510106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61e69-9142-4aab-95b4-c66db110ad09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:57.521318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc61e69-9142-4aab-95b4-c66db110ad09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:57.522857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc61e69-9142-4aab-95b4-c66db110ad09 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:57.535982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1237/2422] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:40:57.564752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febaa50c-5a70-419d-a362-4e3bda17b272 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:57.656452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febaa50c-5a70-419d-a362-4e3bda17b272 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:57.661493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:57.662281 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:57.662622 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:57.662907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] PATH_INFO: `/v3/users/b0d95431c0954828ba83fe5325e31f25` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:57.664238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d95431c0954828ba83fe5325e31f25)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:57.666421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:57.742705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d95431c0954828ba83fe5325e31f25'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:57.751842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d95431c0954828ba83fe5325e31f25'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:57.752695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d95431c0954828ba83fe5325e31f25'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:57.768126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d95431c0954828ba83fe5325e31f25'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:57.769033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d95431c0954828ba83fe5325e31f25'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:57.795556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1186/2423] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:40:57 2022] DELETE /identity/v3/users/b0d95431c0954828ba83fe5325e31f25 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:40:57.800732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03744677-9119-451d-9887-df1b1bb71c89 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:57.801103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03744677-9119-451d-9887-df1b1bb71c89 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:57.801496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03744677-9119-451d-9887-df1b1bb71c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:57.821441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-03744677-9119-451d-9887-df1b1bb71c89 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:40:57.822663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-03744677-9119-451d-9887-df1b1bb71c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:57.869836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:40:57.884685 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1238/2424] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:40:57.895362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:57.897373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:57.955516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2c634-fb95-41b5-b658-31bdf0aee8fa admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:57.959208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250055db-1a15-4c19-a4ba-58aa62bfbfae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:57.959853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250055db-1a15-4c19-a4ba-58aa62bfbfae service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:57.960136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250055db-1a15-4c19-a4ba-58aa62bfbfae service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:57.960410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250055db-1a15-4c19-a4ba-58aa62bfbfae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:57.970532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250055db-1a15-4c19-a4ba-58aa62bfbfae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:57.971986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250055db-1a15-4c19-a4ba-58aa62bfbfae service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:57.981733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1187/2425] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:40:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:40:58.283121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03744677-9119-451d-9887-df1b1bb71c89 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:40:58.284943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-03744677-9119-451d-9887-df1b1bb71c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:40:58.299852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03744677-9119-451d-9887-df1b1bb71c89 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:40:58.317605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:40:58.318640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:40:58.318929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:40:58.319203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] PATH_INFO: `/v3/projects/1643e2f438c54025968e23a4f8fbd7e2` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:40:58.320410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1643e2f438c54025968e23a4f8fbd7e2)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:40:58.322377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:40:58.381161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1643e2f438c54025968e23a4f8fbd7e2', 'request_id': 'req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:58.387523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1643e2f438c54025968e23a4f8fbd7e2', 'request_id': 'req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:58.398774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1643e2f438c54025968e23a4f8fbd7e2', 'request_id': 'req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:58.412599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] The token cache is being invalidate because project 1643e2f438c54025968e23a4f8fbd7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:40:58.413352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:58.413603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6758a725-d8f6-4ad4-8e1f-f8ea72b73190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:40:58.420318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1239/2426] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:40:58 2022] DELETE /identity/v3/projects/1643e2f438c54025968e23a4f8fbd7e2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:41:03.379936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250055db-1a15-4c19-a4ba-58aa62bfbfae service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:41:03.384169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:41:03.389032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250055db-1a15-4c19-a4ba-58aa62bfbfae service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:41:03.448873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:41:03.449565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:41:03.449851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:41:03.450127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:41:03.451915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:41:03.522704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:41:03.525687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:41:03.540324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1188/2427] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:41:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:41:08.408824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:41:13.501174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b855b-ea75-48fe-a7d4-93f57b6de5a1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:41:13.501816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b855b-ea75-48fe-a7d4-93f57b6de5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:41:13.501816 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b855b-ea75-48fe-a7d4-93f57b6de5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:41:13.523371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5f6b855b-ea75-48fe-a7d4-93f57b6de5a1 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:41:13.524575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5f6b855b-ea75-48fe-a7d4-93f57b6de5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:41:13.533499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=49 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:41:13.587907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1240/2428] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:41:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:41:13.600253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:41:13.601728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:41:13.670114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d257e-65fa-419f-9ad8-993371074e36 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:41:13.675324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162e581-1074-4177-b1a8-56c6f20e0500 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:41:13.675948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e581-1074-4177-b1a8-56c6f20e0500 service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:41:13.676233 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e581-1074-4177-b1a8-56c6f20e0500 service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:41:13.676511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e581-1074-4177-b1a8-56c6f20e0500 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:41:13.684538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162e581-1074-4177-b1a8-56c6f20e0500 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:41:13.686147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162e581-1074-4177-b1a8-56c6f20e0500 service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:41:13.695030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1189/2429] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:41:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:41:19.808592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13c4cf-ba19-48e6-ad59-7672614a3f73 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:41:19.809270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13c4cf-ba19-48e6-ad59-7672614a3f73 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:41:19.809270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13c4cf-ba19-48e6-ad59-7672614a3f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:41:19.829862 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7b13c4cf-ba19-48e6-ad59-7672614a3f73 None None] MFA Rules not processed for user `bb9f183ac716439aab70e789f9561901`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:41:19.831104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7b13c4cf-ba19-48e6-ad59-7672614a3f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:41:19.889632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1241/2430] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:41:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:41:23.691553 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=32 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:41:29.882578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=50 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:41:43.442141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98ef3c-aa29-419c-ad5b-f6e3992dad8f None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:41:43.442920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98ef3c-aa29-419c-ad5b-f6e3992dad8f None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:41:43.443034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98ef3c-aa29-419c-ad5b-f6e3992dad8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:41:43.462490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ca98ef3c-aa29-419c-ad5b-f6e3992dad8f None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:41:43.463686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ca98ef3c-aa29-419c-ad5b-f6e3992dad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:41:43.471410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1190/2431] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jul 28 06:41:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:41:43.493532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13c4cf-ba19-48e6-ad59-7672614a3f73 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:41:43.510319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13c4cf-ba19-48e6-ad59-7672614a3f73 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:41:43.515179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:41:43.515820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:41:43.516091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:41:43.516359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:41:43.525090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:41:43.526673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:41:43.535323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1242/2432] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:41:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:41:53.456850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:41:53.531824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:06.885199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98ef3c-aa29-419c-ad5b-f6e3992dad8f None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:06.886717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ca98ef3c-aa29-419c-ad5b-f6e3992dad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:06.898867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98ef3c-aa29-419c-ad5b-f6e3992dad8f None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:06.902406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:06.903013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:06.903299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:06.903576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:06.905268 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:06.975134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:06.976603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:06.987520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1191/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:09.979813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:09.981247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:09.993038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a74732-8557-4569-a073-3a81b7b1ec0e service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:09.996578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea93fad-2eeb-4ddf-8c9a-7c88e2087c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:09.997187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea93fad-2eeb-4ddf-8c9a-7c88e2087c9e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:09.997501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea93fad-2eeb-4ddf-8c9a-7c88e2087c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:09.997776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea93fad-2eeb-4ddf-8c9a-7c88e2087c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:09.999432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5ea93fad-2eeb-4ddf-8c9a-7c88e2087c9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:10.012416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea93fad-2eeb-4ddf-8c9a-7c88e2087c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:10.013879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea93fad-2eeb-4ddf-8c9a-7c88e2087c9e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:10.025466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1243/2434] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:16.982643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=34 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:20.020593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:22.138987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:22.141050 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:22.218591 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1772b5-cbeb-465e-84cf-18c6a54ea367 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:22.223053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:22.223700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:22.223990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:22.224269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:22.226116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:22.303827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:22.305272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:22.316196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1192/2435] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:25.942703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218133-e00c-474b-bdcb-9dd2e9777f09 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:25.943229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218133-e00c-474b-bdcb-9dd2e9777f09 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:25.943338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218133-e00c-474b-bdcb-9dd2e9777f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:25.962468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ed218133-e00c-474b-bdcb-9dd2e9777f09 None None] MFA Rules not processed for user `4a1db92e28444854b9de817dd36104ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:42:25.963666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ed218133-e00c-474b-bdcb-9dd2e9777f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:26.017148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1244/2436] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:42:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:42:26.331443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:26.340641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4cabc-4327-42b9-bf1b-ea135baf5839 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:26.344288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:26.344893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:26.345174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:26.345490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:26.347309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:26.356805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:26.358249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:26.366587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1193/2437] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:26.679235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed218133-e00c-474b-bdcb-9dd2e9777f09 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:26.680864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ed218133-e00c-474b-bdcb-9dd2e9777f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:26.753059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed218133-e00c-474b-bdcb-9dd2e9777f09 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:26.758076 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:26.758688 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:26.758963 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:26.759237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:26.760954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:26.772952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:26.774537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:26.785241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1245/2438] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:28.600914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:28.611205 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e954-7928-4d53-9ccc-d4ffc265e20e service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:28.615879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:28.616564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:28.616830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:28.617108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:28.619244 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:28.630812 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:28.632266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:28.641947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1194/2439] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:36.551976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:36.561874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ef64c-2cb8-46f1-bd5b-8567ef6efe50 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:36.565753 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c803b98-c818-4e16-9048-b172ba52da13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:36.566450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c803b98-c818-4e16-9048-b172ba52da13 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:36.567623 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c803b98-c818-4e16-9048-b172ba52da13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:36.567960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c803b98-c818-4e16-9048-b172ba52da13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:36.568079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:36.569506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:36.570157 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2c803b98-c818-4e16-9048-b172ba52da13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:36.581309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c803b98-c818-4e16-9048-b172ba52da13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:36.583475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c803b98-c818-4e16-9048-b172ba52da13 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:36.587217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e921548-4f18-491a-bc7f-5c433c3b39c0 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:36.593825 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1246/2440] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:36.594401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706afb8f-5f6f-4d3f-be12-b362e63b98aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:36.595097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706afb8f-5f6f-4d3f-be12-b362e63b98aa service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:36.595495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706afb8f-5f6f-4d3f-be12-b362e63b98aa service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:36.595869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706afb8f-5f6f-4d3f-be12-b362e63b98aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:36.597856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-706afb8f-5f6f-4d3f-be12-b362e63b98aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:36.665442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706afb8f-5f6f-4d3f-be12-b362e63b98aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:36.667064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706afb8f-5f6f-4d3f-be12-b362e63b98aa service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:36.677923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1195/2441] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:37.118147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0347eaf-f19a-4c0f-a000-acf335a4bc72 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.118440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0347eaf-f19a-4c0f-a000-acf335a4bc72 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.118718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0347eaf-f19a-4c0f-a000-acf335a4bc72 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.120112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1247/2442] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 06:42:37.150247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f06d3-7342-41dc-b8c2-ae5731b154a0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.150542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f06d3-7342-41dc-b8c2-ae5731b154a0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.150825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f06d3-7342-41dc-b8c2-ae5731b154a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.170982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4c6f06d3-7342-41dc-b8c2-ae5731b154a0 None None] MFA Rules not processed for user `07b18ac45ec948738129f3d08e0d0eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:42:37.172656 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4c6f06d3-7342-41dc-b8c2-ae5731b154a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:37.181547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1196/2443] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:42:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:42:37.186559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0347eaf-f19a-4c0f-a000-acf335a4bc72 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:37.188375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f0347eaf-f19a-4c0f-a000-acf335a4bc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:37.211635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f06d3-7342-41dc-b8c2-ae5731b154a0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:37.222787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f06d3-7342-41dc-b8c2-ae5731b154a0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:37.228772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a8293-1993-44b7-a7a1-dd803f6b5e38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:37.229423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a8293-1993-44b7-a7a1-dd803f6b5e38 service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.229890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a8293-1993-44b7-a7a1-dd803f6b5e38 service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.230170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a8293-1993-44b7-a7a1-dd803f6b5e38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.239027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a8293-1993-44b7-a7a1-dd803f6b5e38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:37.240775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a8293-1993-44b7-a7a1-dd803f6b5e38 service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:37.260868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1197/2444] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:42:37.284287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0347eaf-f19a-4c0f-a000-acf335a4bc72 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:37.290393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:37.291176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.291971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.291971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] PATH_INFO: `/v3/users/bb9f183ac716439aab70e789f9561901` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.293231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9f183ac716439aab70e789f9561901)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:37.295093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:37.327480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:37.362345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9f183ac716439aab70e789f9561901'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.375077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9f183ac716439aab70e789f9561901'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.386035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9f183ac716439aab70e789f9561901'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.386711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9f183ac716439aab70e789f9561901'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.401367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-87ba49a4-4343-40a0-b014-67c6763b6589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9f183ac716439aab70e789f9561901'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.408190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1248/2445] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:42:37 2022] DELETE /identity/v3/users/bb9f183ac716439aab70e789f9561901 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:37.588521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a8293-1993-44b7-a7a1-dd803f6b5e38 service placement] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:37.604152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a8293-1993-44b7-a7a1-dd803f6b5e38 service placement] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:37.608054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:37.665306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:37.665958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.666240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.666513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] PATH_INFO: `/v3/projects/bafc15c173f94f8eb804e56a2d4673a8` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.667655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bafc15c173f94f8eb804e56a2d4673a8)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:37.669605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:37.715265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bafc15c173f94f8eb804e56a2d4673a8', 'request_id': 'req-70db44e8-22ae-445f-8953-94ce40b7c9d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.723617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bafc15c173f94f8eb804e56a2d4673a8', 'request_id': 'req-70db44e8-22ae-445f-8953-94ce40b7c9d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.729347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bafc15c173f94f8eb804e56a2d4673a8', 'request_id': 'req-70db44e8-22ae-445f-8953-94ce40b7c9d9'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.741786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] The token cache is being invalidate because project bafc15c173f94f8eb804e56a2d4673a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:42:37.742119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.743054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-70db44e8-22ae-445f-8953-94ce40b7c9d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:37.745028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1198/2446] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:42:37 2022] DELETE /identity/v3/projects/bafc15c173f94f8eb804e56a2d4673a8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:37.749960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c2e9-cfd8-4e4b-b18b-86b3862b09fd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.750438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c2e9-cfd8-4e4b-b18b-86b3862b09fd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.750578 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c2e9-cfd8-4e4b-b18b-86b3862b09fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.770538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5fd5c2e9-cfd8-4e4b-b18b-86b3862b09fd None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:42:37.771766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5fd5c2e9-cfd8-4e4b-b18b-86b3862b09fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:37.832716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1249/2447] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:42:37.840996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac7e85-358a-4101-b5e0-ed303dfa8ac6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.841285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac7e85-358a-4101-b5e0-ed303dfa8ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.841584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac7e85-358a-4101-b5e0-ed303dfa8ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.862567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d1ac7e85-358a-4101-b5e0-ed303dfa8ac6 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:42:37.864416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d1ac7e85-358a-4101-b5e0-ed303dfa8ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:37.872527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1199/2448] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:42:37.876668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c2e9-cfd8-4e4b-b18b-86b3862b09fd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:37.886777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c2e9-cfd8-4e4b-b18b-86b3862b09fd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:37.891349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:37.891957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.892236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.892508 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.893200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:42:37.893538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:37.895657 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:37.907008 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1250/2449] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:42:37.911778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac7e85-358a-4101-b5e0-ed303dfa8ac6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:37.921184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac7e85-358a-4101-b5e0-ed303dfa8ac6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:37.925792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1519fbf-9eb7-43a4-bd5d-bca932f1ca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:37.927438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1519fbf-9eb7-43a4-bd5d-bca932f1ca86 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:37.927722 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1519fbf-9eb7-43a4-bd5d-bca932f1ca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:37.928015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1519fbf-9eb7-43a4-bd5d-bca932f1ca86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:37.928658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1519fbf-9eb7-43a4-bd5d-bca932f1ca86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:37.930575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1519fbf-9eb7-43a4-bd5d-bca932f1ca86 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:37.957829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1200/2450] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:42:37 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:42:37.961528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:37.971307 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264d23e-8b80-4ec7-a499-abbdba81dd7c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:38.036083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b602c0-7fee-4bc8-83a6-ee1001c56770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:38.037246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b602c0-7fee-4bc8-83a6-ee1001c56770 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:38.037558 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b602c0-7fee-4bc8-83a6-ee1001c56770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:38.037828 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b602c0-7fee-4bc8-83a6-ee1001c56770 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:38.038525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b602c0-7fee-4bc8-83a6-ee1001c56770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:38.040448 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b602c0-7fee-4bc8-83a6-ee1001c56770 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:38.064263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1251/2451] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:42:37 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:42:38.068738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1519fbf-9eb7-43a4-bd5d-bca932f1ca86 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:38.079482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1519fbf-9eb7-43a4-bd5d-bca932f1ca86 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:38.088174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73370f-8a93-4faf-b7e9-6162b5b1f627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:38.088720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73370f-8a93-4faf-b7e9-6162b5b1f627 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:38.089003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73370f-8a93-4faf-b7e9-6162b5b1f627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:38.089278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73370f-8a93-4faf-b7e9-6162b5b1f627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:38.090007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73370f-8a93-4faf-b7e9-6162b5b1f627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:38.091595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73370f-8a93-4faf-b7e9-6162b5b1f627 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:38.102829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1201/2452] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:42:38.106785 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b602c0-7fee-4bc8-83a6-ee1001c56770 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:38.117282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b602c0-7fee-4bc8-83a6-ee1001c56770 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:38.121802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa52ab-de6d-4e85-a337-81a1f2cd9bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:38.122404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa52ab-de6d-4e85-a337-81a1f2cd9bac admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:38.122680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa52ab-de6d-4e85-a337-81a1f2cd9bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:38.122947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa52ab-de6d-4e85-a337-81a1f2cd9bac admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:38.123631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa52ab-de6d-4e85-a337-81a1f2cd9bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:38.125195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa52ab-de6d-4e85-a337-81a1f2cd9bac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:38.135028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1252/2453] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:42:38.139116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73370f-8a93-4faf-b7e9-6162b5b1f627 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:38.149662 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73370f-8a93-4faf-b7e9-6162b5b1f627 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:38.154131 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:38.154780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:38.155061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:38.155336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] PATH_INFO: `/v3/projects/b73764cba45b491586d48b7d67c637f4/users/c5913d21f8234e44a3056c1f90923ae0/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:38.170419 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73764cba45b491586d48b7d67c637f4, user_id=c5913d21f8234e44a3056c1f90923ae0, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:38.173038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:38.186223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1202/2454] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:42:38 2022] PUT /identity/v3/projects/b73764cba45b491586d48b7d67c637f4/users/c5913d21f8234e44a3056c1f90923ae0/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:38.196466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61498f-b10c-4901-82ab-9892e0826fc5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:38.196752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61498f-b10c-4901-82ab-9892e0826fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:38.197025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61498f-b10c-4901-82ab-9892e0826fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:38.212995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:38.231232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7a61498f-b10c-4901-82ab-9892e0826fc5 None None] MFA Rules not processed for user `c5913d21f8234e44a3056c1f90923ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:42:38.232532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7a61498f-b10c-4901-82ab-9892e0826fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:38.310712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1253/2455] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:42:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:42:38.323084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:38.324772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:38.360028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:38.383397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52d9c1-c9e3-4ca0-b998-403233178695 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:38.387077 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331d248-d8b9-444e-bf0d-5665c99b0b35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:38.387692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331d248-d8b9-444e-bf0d-5665c99b0b35 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:38.387975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331d248-d8b9-444e-bf0d-5665c99b0b35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:38.388248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331d248-d8b9-444e-bf0d-5665c99b0b35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:38.396721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331d248-d8b9-444e-bf0d-5665c99b0b35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:38.398181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331d248-d8b9-444e-bf0d-5665c99b0b35 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:38.407206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1203/2456] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:45.143631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61498f-b10c-4901-82ab-9892e0826fc5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:45.146359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7a61498f-b10c-4901-82ab-9892e0826fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:45.248774 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61498f-b10c-4901-82ab-9892e0826fc5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:45.257422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12678dd6-eb39-4981-bdde-aae21a17b102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:45.258138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678dd6-eb39-4981-bdde-aae21a17b102 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:45.258422 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678dd6-eb39-4981-bdde-aae21a17b102 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:45.258761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678dd6-eb39-4981-bdde-aae21a17b102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:45.261487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-12678dd6-eb39-4981-bdde-aae21a17b102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:45.278804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12678dd6-eb39-4981-bdde-aae21a17b102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:45.281725 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12678dd6-eb39-4981-bdde-aae21a17b102 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:45.297067 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1254/2457] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:46.640935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331d248-d8b9-444e-bf0d-5665c99b0b35 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:46.706256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331d248-d8b9-444e-bf0d-5665c99b0b35 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:46.712030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:46.712680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:46.712967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:46.713246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] PATH_INFO: `/v3/users/c5913d21f8234e44a3056c1f90923ae0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:46.714539 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5913d21f8234e44a3056c1f90923ae0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:46.716504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:46.768820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5913d21f8234e44a3056c1f90923ae0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:46.778292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5913d21f8234e44a3056c1f90923ae0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:46.779115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5913d21f8234e44a3056c1f90923ae0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:46.797496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5913d21f8234e44a3056c1f90923ae0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:46.798431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5913d21f8234e44a3056c1f90923ae0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:46.813398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1204/2458] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:42:46 2022] DELETE /identity/v3/users/c5913d21f8234e44a3056c1f90923ae0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:46.821611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ddaf-6237-4c5e-a089-3b8f6255f14c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:46.822553 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ddaf-6237-4c5e-a089-3b8f6255f14c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:46.823563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ddaf-6237-4c5e-a089-3b8f6255f14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:46.847108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fbf6ddaf-6237-4c5e-a089-3b8f6255f14c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:42:46.849822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fbf6ddaf-6237-4c5e-a089-3b8f6255f14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:46.919671 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1255/2459] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:42:46.933093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:46.934734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:47.015340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa559b65-5204-4b41-a3de-961895811849 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:47.019008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e5eec-f3c2-459c-b8ce-59691ba68bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:47.019627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e5eec-f3c2-459c-b8ce-59691ba68bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:47.019907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e5eec-f3c2-459c-b8ce-59691ba68bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:47.020975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e5eec-f3c2-459c-b8ce-59691ba68bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:47.031734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e5eec-f3c2-459c-b8ce-59691ba68bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:47.033228 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e5eec-f3c2-459c-b8ce-59691ba68bfc service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:47.038122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:47.045585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1205/2460] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:42:47.415324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ddaf-6237-4c5e-a089-3b8f6255f14c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:47.418096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fbf6ddaf-6237-4c5e-a089-3b8f6255f14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:47.443338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ddaf-6237-4c5e-a089-3b8f6255f14c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:47.448769 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:47.449464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:47.449752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:47.450030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] PATH_INFO: `/v3/projects/b73764cba45b491586d48b7d67c637f4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:47.451228 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73764cba45b491586d48b7d67c637f4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:47.453780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:47.472319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:42:47.508576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73764cba45b491586d48b7d67c637f4', 'request_id': 'req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:47.515433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73764cba45b491586d48b7d67c637f4', 'request_id': 'req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:47.524255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73764cba45b491586d48b7d67c637f4', 'request_id': 'req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:47.540809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] The token cache is being invalidate because project b73764cba45b491586d48b7d67c637f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:42:47.541112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:47.542273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-59e22cc7-c05d-4d1a-aac8-cd9da149dfa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:42:47.544588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1256/2461] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:42:47 2022] DELETE /identity/v3/projects/b73764cba45b491586d48b7d67c637f4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:42:48.600213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a36a-1dc6-4558-8d9f-9a383a1e47c7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:48.600702 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a36a-1dc6-4558-8d9f-9a383a1e47c7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:48.600823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a36a-1dc6-4558-8d9f-9a383a1e47c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:48.629125 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-3b49a36a-1dc6-4558-8d9f-9a383a1e47c7 None None] MFA Rules not processed for user `cf13ae714ad04d6696c273603bba9fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:42:48.630364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3b49a36a-1dc6-4558-8d9f-9a383a1e47c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:48.684395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1206/2462] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:42:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:42:51.755040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989b9a-3759-4105-9046-77d3649e0a54 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:51.755632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989b9a-3759-4105-9046-77d3649e0a54 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:51.756518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989b9a-3759-4105-9046-77d3649e0a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:51.779096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-05989b9a-3759-4105-9046-77d3649e0a54 None None] MFA Rules not processed for user `a2080efc6cbd471995593012bc268765`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:42:51.780316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-05989b9a-3759-4105-9046-77d3649e0a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:51.836290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1257/2463] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:42:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:42:57.594078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49a36a-1dc6-4558-8d9f-9a383a1e47c7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:42:57.596354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3b49a36a-1dc6-4558-8d9f-9a383a1e47c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:57.719751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49a36a-1dc6-4558-8d9f-9a383a1e47c7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:42:57.727850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae35fa-f16a-49fa-872a-18362b7d1636 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:42:57.728857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae35fa-f16a-49fa-872a-18362b7d1636 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:42:57.729401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae35fa-f16a-49fa-872a-18362b7d1636 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:42:57.729886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae35fa-f16a-49fa-872a-18362b7d1636 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:42:57.732634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-14ae35fa-f16a-49fa-872a-18362b7d1636 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:42:57.749312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae35fa-f16a-49fa-872a-18362b7d1636 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:42:57.751345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae35fa-f16a-49fa-872a-18362b7d1636 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:42:57.773163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1207/2464] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:42:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:43:01.828614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:43:02.689651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05989b9a-3759-4105-9046-77d3649e0a54 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:43:02.712627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05989b9a-3759-4105-9046-77d3649e0a54 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:43:02.725523 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d15f2-af77-4ab3-bbe9-f12909f311e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:43:02.726537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d15f2-af77-4ab3-bbe9-f12909f311e7 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:02.727665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d15f2-af77-4ab3-bbe9-f12909f311e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:02.728119 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d15f2-af77-4ab3-bbe9-f12909f311e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:02.730918 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e29d15f2-af77-4ab3-bbe9-f12909f311e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:02.751083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d15f2-af77-4ab3-bbe9-f12909f311e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:43:02.753387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d15f2-af77-4ab3-bbe9-f12909f311e7 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:43:02.767850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1258/2465] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:43:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:43:07.556371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c5f65-02f8-4c18-aaf3-01787647608c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:07.556849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c5f65-02f8-4c18-aaf3-01787647608c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:07.556950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c5f65-02f8-4c18-aaf3-01787647608c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:07.576520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f60c5f65-02f8-4c18-aaf3-01787647608c None None] MFA Rules not processed for user `13ca4be686f84c279f87016c20129833`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:43:07.577753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f60c5f65-02f8-4c18-aaf3-01787647608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:07.641171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1208/2466] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:43:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:43:09.153669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c425a6-82e0-48f0-a7f4-5862bff479ad None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:09.154202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c425a6-82e0-48f0-a7f4-5862bff479ad None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:09.154418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c425a6-82e0-48f0-a7f4-5862bff479ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:09.176395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-86c425a6-82e0-48f0-a7f4-5862bff479ad None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:43:09.178409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-86c425a6-82e0-48f0-a7f4-5862bff479ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:09.245964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1259/2467] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jul 28 06:43:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:43:09.313998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c5f65-02f8-4c18-aaf3-01787647608c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:43:09.315762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f60c5f65-02f8-4c18-aaf3-01787647608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:09.356195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:43:09.394747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c5f65-02f8-4c18-aaf3-01787647608c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:43:09.401070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:43:09.402038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:09.402545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:09.402911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:09.414018 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:43:09.415763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:43:09.428416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1209/2468] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:43:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:43:15.230326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c425a6-82e0-48f0-a7f4-5862bff479ad None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:43:15.231781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-86c425a6-82e0-48f0-a7f4-5862bff479ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:15.307944 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c425a6-82e0-48f0-a7f4-5862bff479ad None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:43:15.313444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:43:15.313986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:15.314271 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:15.314575 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:15.316339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:15.329246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:43:15.330869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:43:15.341164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1260/2469] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:43:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:43:18.609791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:43:18.612379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:18.685821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e12e7d-31df-4997-a616-b921d7bf4eb7 service swift] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:43:18.690563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:43:18.691210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:18.691496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:18.691775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] PATH_INFO: `/v3/users/93dacbaabf7343bbae337761b8d8cd41` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:18.693582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] RBAC: Authorizing `identity:delete_user(user_id=93dacbaabf7343bbae337761b8d8cd41)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:43:18.695541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:43:18.745540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93dacbaabf7343bbae337761b8d8cd41'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:18.753160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93dacbaabf7343bbae337761b8d8cd41'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:18.753848 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93dacbaabf7343bbae337761b8d8cd41'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:18.767128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93dacbaabf7343bbae337761b8d8cd41'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:18.767778 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93dacbaabf7343bbae337761b8d8cd41'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:18.780930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1210/2470] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:43:18 2022] DELETE /identity/v3/users/93dacbaabf7343bbae337761b8d8cd41 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:43:18.961352 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:43:18.972044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560f4ab-b959-4c25-b35a-48faf8edc738 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:43:19.029078 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:43:19.029746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:19.030028 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:19.030302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] PATH_INFO: `/v3/projects/6b7e4c91aaaf48e2b134128055919a47` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:19.031525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b7e4c91aaaf48e2b134128055919a47)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:43:19.033394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:43:19.059479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:43:19.082902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b7e4c91aaaf48e2b134128055919a47', 'request_id': 'req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.088188 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b7e4c91aaaf48e2b134128055919a47', 'request_id': 'req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.095370 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b7e4c91aaaf48e2b134128055919a47', 'request_id': 'req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.106916 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] The token cache is being invalidate because project 6b7e4c91aaaf48e2b134128055919a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:43:19.107203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.107831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.109414 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1261/2471] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:43:18 2022] DELETE /identity/v3/projects/6b7e4c91aaaf48e2b134128055919a47 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:43:19.113238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:43:19.115305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:19.128735 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7c2c-dfb0-47ef-bbb5-30c31367d87a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:43:19.169106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:43:19.186618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:43:19.187262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:19.187569 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:19.187870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] PATH_INFO: `/v3/users/0bef5e8c46f2481eb433383cd8c6e0a5` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:19.189061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bef5e8c46f2481eb433383cd8c6e0a5)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:43:19.191005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:43:19.239562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bef5e8c46f2481eb433383cd8c6e0a5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.247221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bef5e8c46f2481eb433383cd8c6e0a5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.247869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bef5e8c46f2481eb433383cd8c6e0a5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.260401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bef5e8c46f2481eb433383cd8c6e0a5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.261039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8b820ffd-b7d7-4b6a-b5f3-d0f53dc42258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bef5e8c46f2481eb433383cd8c6e0a5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.273657 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1211/2472] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:43:19 2022] DELETE /identity/v3/users/0bef5e8c46f2481eb433383cd8c6e0a5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:43:19.450536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:43:19.451668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:19.464827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f5632-d3f4-417b-9df4-f4b4c86aa3d0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:43:19.520889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:43:19.521560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:19.521841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:19.522114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] PATH_INFO: `/v3/projects/e44ec68574a94e0ebc807f38d47b76cb` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:19.523452 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e44ec68574a94e0ebc807f38d47b76cb)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:43:19.525369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:43:19.572360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e44ec68574a94e0ebc807f38d47b76cb', 'request_id': 'req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.579198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e44ec68574a94e0ebc807f38d47b76cb', 'request_id': 'req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.586598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e44ec68574a94e0ebc807f38d47b76cb', 'request_id': 'req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.597988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] The token cache is being invalidate because project e44ec68574a94e0ebc807f38d47b76cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:43:19.598813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.600083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:43:19.601838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1262/2473] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:43:19 2022] DELETE /identity/v3/projects/e44ec68574a94e0ebc807f38d47b76cb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:43:29.269575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:43:29.594815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:43:40.849703 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5896e-ab34-48bf-b817-5081168f238b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:40.850206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5896e-ab34-48bf-b817-5081168f238b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:40.850313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5896e-ab34-48bf-b817-5081168f238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:40.870305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c5c5896e-ab34-48bf-b817-5081168f238b None None] MFA Rules not processed for user `4a1db92e28444854b9de817dd36104ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:43:40.871505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c5c5896e-ab34-48bf-b817-5081168f238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:40.925733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1212/2474] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:43:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:43:50.918982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=25 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:43:59.416756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:43:59.418246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:59.484431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52405c5-a2cd-40cb-a3f5-1f1e2696fbfb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:43:59.488084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:43:59.488702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:43:59.488983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:43:59.489255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:43:59.491037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:43:59.561968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:43:59.563423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:43:59.574905 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1263/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:44:03.174426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5896e-ab34-48bf-b817-5081168f238b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:03.175901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c5c5896e-ab34-48bf-b817-5081168f238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:03.191051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5896e-ab34-48bf-b817-5081168f238b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:03.195775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:03.196449 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:03.196887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:03.197165 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] PATH_INFO: `/v3/users/a2080efc6cbd471995593012bc268765` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:03.198351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2080efc6cbd471995593012bc268765)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:03.200243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:03.248494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2080efc6cbd471995593012bc268765'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.255733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2080efc6cbd471995593012bc268765'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.256343 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2080efc6cbd471995593012bc268765'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.268512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2080efc6cbd471995593012bc268765'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.269156 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e0e26b14-ff1d-4991-b9f6-055e6ce6805d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2080efc6cbd471995593012bc268765'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.282780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1213/2476] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:44:03 2022] DELETE /identity/v3/users/a2080efc6cbd471995593012bc268765 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:44:03.456864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:03.469026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a8015-bcb3-4fe7-b198-bf484c65d2d3 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:03.529309 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:03.529959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:03.530248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:03.530563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] PATH_INFO: `/v3/projects/0fb2f4ad616e43f5a9a248390bd0aebf` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:03.531718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb2f4ad616e43f5a9a248390bd0aebf)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:03.533653 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:03.590338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb2f4ad616e43f5a9a248390bd0aebf', 'request_id': 'req-1995b308-9fa6-4b2b-8255-b015332e7b50'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.592378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:03.598152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb2f4ad616e43f5a9a248390bd0aebf', 'request_id': 'req-1995b308-9fa6-4b2b-8255-b015332e7b50'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.609722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb2f4ad616e43f5a9a248390bd0aebf', 'request_id': 'req-1995b308-9fa6-4b2b-8255-b015332e7b50'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.622503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] The token cache is being invalidate because project 0fb2f4ad616e43f5a9a248390bd0aebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:44:03.622882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.623597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1995b308-9fa6-4b2b-8255-b015332e7b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:44:03.625172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1264/2477] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:44:03 2022] DELETE /identity/v3/projects/0fb2f4ad616e43f5a9a248390bd0aebf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:44:03.630080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da38138-7375-42c5-851e-2d1640947290 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:03.630470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da38138-7375-42c5-851e-2d1640947290 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:03.630968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da38138-7375-42c5-851e-2d1640947290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:03.653382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6da38138-7375-42c5-851e-2d1640947290 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:03.654678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6da38138-7375-42c5-851e-2d1640947290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:03.842129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1214/2478] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:44:03.850618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98f374-51bc-42ca-a615-71ed780939a7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:03.850989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98f374-51bc-42ca-a615-71ed780939a7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:03.851326 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98f374-51bc-42ca-a615-71ed780939a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:03.873990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fa98f374-51bc-42ca-a615-71ed780939a7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:03.875210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fa98f374-51bc-42ca-a615-71ed780939a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:03.883140 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1265/2479] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:44:03.888645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da38138-7375-42c5-851e-2d1640947290 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:03.902884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da38138-7375-42c5-851e-2d1640947290 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:03.907859 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:03.908527 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:03.908831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:03.909107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:03.909835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:44:03.910141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:03.911724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:03.923551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1215/2480] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:44:03.927924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98f374-51bc-42ca-a615-71ed780939a7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:03.937272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98f374-51bc-42ca-a615-71ed780939a7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:03.942204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09378d-18ec-4b44-9836-5e94f3382902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:03.942808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09378d-18ec-4b44-9836-5e94f3382902 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:03.943114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09378d-18ec-4b44-9836-5e94f3382902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:03.943364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09378d-18ec-4b44-9836-5e94f3382902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:03.944017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09378d-18ec-4b44-9836-5e94f3382902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:03.946691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09378d-18ec-4b44-9836-5e94f3382902 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:03.979646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1266/2481] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:44:03 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:44:03.983932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:03.994962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c874b5d-4abb-4343-a3e4-086685fda38f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:04.054724 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7088ec2-ee46-459b-9b61-6a9bc2aa2bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:04.055373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7088ec2-ee46-459b-9b61-6a9bc2aa2bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:04.055653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7088ec2-ee46-459b-9b61-6a9bc2aa2bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:04.055919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7088ec2-ee46-459b-9b61-6a9bc2aa2bcc admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:04.056581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7088ec2-ee46-459b-9b61-6a9bc2aa2bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:04.058494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7088ec2-ee46-459b-9b61-6a9bc2aa2bcc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:04.066344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:04.084806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1216/2482] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:44:03 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:44:04.089931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09378d-18ec-4b44-9836-5e94f3382902 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:04.100778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09378d-18ec-4b44-9836-5e94f3382902 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:04.105700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1646-ef4f-4645-97dc-e28351036bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:04.106318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1646-ef4f-4645-97dc-e28351036bab admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:04.106731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1646-ef4f-4645-97dc-e28351036bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:04.106846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1646-ef4f-4645-97dc-e28351036bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:04.107652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d1646-ef4f-4645-97dc-e28351036bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:04.109230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d1646-ef4f-4645-97dc-e28351036bab admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:04.121348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1267/2483] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:44:04.123841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7088ec2-ee46-459b-9b61-6a9bc2aa2bcc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:04.134718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7088ec2-ee46-459b-9b61-6a9bc2aa2bcc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:04.139306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76dc54-1635-4e35-93a1-9bfd800ba048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:04.139909 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76dc54-1635-4e35-93a1-9bfd800ba048 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:04.140191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76dc54-1635-4e35-93a1-9bfd800ba048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:04.140464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76dc54-1635-4e35-93a1-9bfd800ba048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:04.141154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76dc54-1635-4e35-93a1-9bfd800ba048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:04.142766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76dc54-1635-4e35-93a1-9bfd800ba048 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:04.153951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1217/2484] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:44:04.158045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1646-ef4f-4645-97dc-e28351036bab admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:04.167980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1646-ef4f-4645-97dc-e28351036bab admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:04.172530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:04.173184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:04.173505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:04.173788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] PATH_INFO: `/v3/projects/f0f65f082a2547f79ac218e318bee171/users/a87b79d20f0f4194a99fc342c9b14aa2/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:04.188179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f65f082a2547f79ac218e318bee171, user_id=a87b79d20f0f4194a99fc342c9b14aa2, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:04.190958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:04.204990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1268/2485] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:44:04 2022] PUT /identity/v3/projects/f0f65f082a2547f79ac218e318bee171/users/a87b79d20f0f4194a99fc342c9b14aa2/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:44:04.210260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad640696-721c-4a59-a6fa-7137a52535de None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:04.210775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad640696-721c-4a59-a6fa-7137a52535de None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:04.210885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad640696-721c-4a59-a6fa-7137a52535de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:04.230667 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-ad640696-721c-4a59-a6fa-7137a52535de None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:04.231852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-ad640696-721c-4a59-a6fa-7137a52535de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:04.291283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1218/2486] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:44:04.301571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:04.303246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:04.361055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae4534-07c8-4c7b-8293-9cc1fd0e72c8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:04.364655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:04.365272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:04.365589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:04.365864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:04.376666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:04.378120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:04.388304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1269/2487] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:44:08.576187 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5813f-77bc-439e-bf4a-3f764bcc4ce7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:08.576649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5813f-77bc-439e-bf4a-3f764bcc4ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:08.576920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5813f-77bc-439e-bf4a-3f764bcc4ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:08.610532 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4fb5813f-77bc-439e-bf4a-3f764bcc4ce7 None None] MFA Rules not processed for user `a87b79d20f0f4194a99fc342c9b14aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:08.611454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4fb5813f-77bc-439e-bf4a-3f764bcc4ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:08.699269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1219/2488] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:44:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:44:08.714149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:08.715667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:08.796314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d1b9d-d245-4904-866b-0df37f8710d1 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:08.801874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021195a1-146c-4da3-9d84-466ca0304242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:08.802505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021195a1-146c-4da3-9d84-466ca0304242 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:08.802793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021195a1-146c-4da3-9d84-466ca0304242 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:08.803068 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021195a1-146c-4da3-9d84-466ca0304242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:08.812469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021195a1-146c-4da3-9d84-466ca0304242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:08.814100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021195a1-146c-4da3-9d84-466ca0304242 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:08.825270 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1270/2489] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:44:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:44:10.264747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d51b4-407a-4419-918f-cc9022fb41a3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:10.265381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d51b4-407a-4419-918f-cc9022fb41a3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:10.265381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d51b4-407a-4419-918f-cc9022fb41a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:10.286241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-af5d51b4-407a-4419-918f-cc9022fb41a3 None None] MFA Rules not processed for user `a87b79d20f0f4194a99fc342c9b14aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:10.287670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-af5d51b4-407a-4419-918f-cc9022fb41a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:10.297333 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1220/2490] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:44:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:44:15.921636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddb55d-67ab-4b96-a208-f59a42bb47fe None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:15.922158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddb55d-67ab-4b96-a208-f59a42bb47fe None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:15.922264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddb55d-67ab-4b96-a208-f59a42bb47fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:15.931311 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:15.943713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-12ddb55d-67ab-4b96-a208-f59a42bb47fe None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:15.945082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-12ddb55d-67ab-4b96-a208-f59a42bb47fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:16.003020 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1271/2491] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:44:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:44:16.052634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d51b4-407a-4419-918f-cc9022fb41a3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:16.054111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-af5d51b4-407a-4419-918f-cc9022fb41a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:16.120964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:16.126102 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d51b4-407a-4419-918f-cc9022fb41a3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:16.132027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d888c97-dab4-4b6c-b5f8-2dbcf92ef2f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:16.132643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d888c97-dab4-4b6c-b5f8-2dbcf92ef2f3 service swift] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:16.133525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d888c97-dab4-4b6c-b5f8-2dbcf92ef2f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:16.133797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d888c97-dab4-4b6c-b5f8-2dbcf92ef2f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:16.142070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d888c97-dab4-4b6c-b5f8-2dbcf92ef2f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:16.143629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d888c97-dab4-4b6c-b5f8-2dbcf92ef2f3 service swift] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:16.152259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1221/2492] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:44:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:44:25.995993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=23 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:26.148898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:29.849277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11327a-ffcd-4372-a3d8-62ab0b3aed15 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:29.849856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11327a-ffcd-4372-a3d8-62ab0b3aed15 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:29.849856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11327a-ffcd-4372-a3d8-62ab0b3aed15 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:29.851216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1272/2493] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:44:29.857225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270029c-d625-427a-ae53-daae5bba46ef None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:29.857751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270029c-d625-427a-ae53-daae5bba46ef None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:29.857853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270029c-d625-427a-ae53-daae5bba46ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:29.878070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d270029c-d625-427a-ae53-daae5bba46ef None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:29.879266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d270029c-d625-427a-ae53-daae5bba46ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:29.889718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1222/2494] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:44:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:44:29.900912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11327a-ffcd-4372-a3d8-62ab0b3aed15 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:29.902824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4d11327a-ffcd-4372-a3d8-62ab0b3aed15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:29.962547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11327a-ffcd-4372-a3d8-62ab0b3aed15 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:29.966117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa8f7d-07d7-45a0-ac01-513394c3f5f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '649334bb7ed04bd69eb971fc070d96e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:29.966732 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa8f7d-07d7-45a0-ac01-513394c3f5f5 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:29.967015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa8f7d-07d7-45a0-ac01-513394c3f5f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:29.967286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa8f7d-07d7-45a0-ac01-513394c3f5f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:29.981830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa8f7d-07d7-45a0-ac01-513394c3f5f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:29.983924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa8f7d-07d7-45a0-ac01-513394c3f5f5 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:29.999915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1273/2495] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:44:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:44:30.067374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0a84b-5311-4b52-b128-db19627cb62e None None] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:30.067594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0a84b-5311-4b52-b128-db19627cb62e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:30.067868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0a84b-5311-4b52-b128-db19627cb62e None None] PATH_INFO: `/` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:30.069238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1223/2496] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:44:30.073923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48034899-ef84-41ea-b280-367d44ed7dc4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:30.074209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48034899-ef84-41ea-b280-367d44ed7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:30.074483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48034899-ef84-41ea-b280-367d44ed7dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:30.095263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-48034899-ef84-41ea-b280-367d44ed7dc4 None None] MFA Rules not processed for user `4b46a15d481f4393bef8c4fa2281710b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:30.096474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-48034899-ef84-41ea-b280-367d44ed7dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:30.104387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1274/2497] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:44:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:44:30.115916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0a84b-5311-4b52-b128-db19627cb62e None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:30.117608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d4c0a84b-5311-4b52-b128-db19627cb62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:30.131907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0a84b-5311-4b52-b128-db19627cb62e None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:30.136211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2b262-1433-4c80-89e8-49e549c96c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:30.136879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2b262-1433-4c80-89e8-49e549c96c0c service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:30.137169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2b262-1433-4c80-89e8-49e549c96c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:30.137491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2b262-1433-4c80-89e8-49e549c96c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:30.144910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2b262-1433-4c80-89e8-49e549c96c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:30.146348 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2b262-1433-4c80-89e8-49e549c96c0c service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:30.154554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1224/2498] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:44:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:44:30.237348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a94e0a-9768-4ea6-ba17-8c2348154913 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:30.237650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a94e0a-9768-4ea6-ba17-8c2348154913 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:30.237927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a94e0a-9768-4ea6-ba17-8c2348154913 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:30.239320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1275/2499] 10.222.0.20 () {54 vars in 921 bytes} [Thu Jul 28 06:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 06:44:30.244993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a084fc8-b5d8-4281-ba84-55d0032dcea5 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:30.245277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a084fc8-b5d8-4281-ba84-55d0032dcea5 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:30.245598 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a084fc8-b5d8-4281-ba84-55d0032dcea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:30.265381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6a084fc8-b5d8-4281-ba84-55d0032dcea5 None None] MFA Rules not processed for user `9f558687eefe4a26beb7d69151221660`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:30.266590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6a084fc8-b5d8-4281-ba84-55d0032dcea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:30.276566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1225/2500] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Jul 28 06:44:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:44:30.287901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a94e0a-9768-4ea6-ba17-8c2348154913 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:30.299269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a94e0a-9768-4ea6-ba17-8c2348154913 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:30.302796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc08e3-af7a-4336-a44f-e1ba9bbce8b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:30.303394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc08e3-af7a-4336-a44f-e1ba9bbce8b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:30.303672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc08e3-af7a-4336-a44f-e1ba9bbce8b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:30.303969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc08e3-af7a-4336-a44f-e1ba9bbce8b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:30.314866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc08e3-af7a-4336-a44f-e1ba9bbce8b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:30.316297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc08e3-af7a-4336-a44f-e1ba9bbce8b5 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:30.327863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1276/2501] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:44:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:44:32.148875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b068ab-a817-4567-9f9a-6a4678138f76 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:32.149908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b068ab-a817-4567-9f9a-6a4678138f76 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:32.150043 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b068ab-a817-4567-9f9a-6a4678138f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:32.169966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-39b068ab-a817-4567-9f9a-6a4678138f76 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:32.171143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-39b068ab-a817-4567-9f9a-6a4678138f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:32.178044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1226/2502] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:44:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:44:32.189756 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc08e3-af7a-4336-a44f-e1ba9bbce8b5 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:32.202801 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc08e3-af7a-4336-a44f-e1ba9bbce8b5 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:32.207767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc8d7b-e390-4408-91df-8c10538440e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:32.208378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8d7b-e390-4408-91df-8c10538440e6 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:32.208665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8d7b-e390-4408-91df-8c10538440e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:32.208937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8d7b-e390-4408-91df-8c10538440e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:32.217983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc8d7b-e390-4408-91df-8c10538440e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:32.219560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc8d7b-e390-4408-91df-8c10538440e6 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:32.228387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1277/2503] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:44:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:44:42.164555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=15 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:42.224477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=41 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:48.517855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057eba6-25fb-4ee9-b5c2-8982f2255ee0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:48.518332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057eba6-25fb-4ee9-b5c2-8982f2255ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:48.518430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057eba6-25fb-4ee9-b5c2-8982f2255ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:48.537178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5057eba6-25fb-4ee9-b5c2-8982f2255ee0 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:44:48.538383 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5057eba6-25fb-4ee9-b5c2-8982f2255ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:44:48.545539 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1227/2504] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jul 28 06:44:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:44:48.557654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc8d7b-e390-4408-91df-8c10538440e6 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:44:48.568961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc8d7b-e390-4408-91df-8c10538440e6 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:44:48.573962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:44:48.574568 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:44:48.574848 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:44:48.575120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:44:48.583781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:44:48.585374 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:44:48.593847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1278/2505] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:44:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:44:58.532189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:44:58.590401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:03.162792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057eba6-25fb-4ee9-b5c2-8982f2255ee0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:03.183286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057eba6-25fb-4ee9-b5c2-8982f2255ee0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:03.188423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:03.189109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:03.189490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:03.189670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:03.191737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:03.259982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:03.261467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:03.281764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1228/2506] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:06.446193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:06.447733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:06.490817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f2152-e55f-499f-a2fd-7f5d86abd27d service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:06.495349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:06.495980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:06.496260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:06.496539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] PATH_INFO: `/v3/users/4a1db92e28444854b9de817dd36104ea` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:06.498423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a1db92e28444854b9de817dd36104ea)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:06.500217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:06.545912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a1db92e28444854b9de817dd36104ea'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.552691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a1db92e28444854b9de817dd36104ea'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.559378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a1db92e28444854b9de817dd36104ea'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.560676 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a1db92e28444854b9de817dd36104ea'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.573288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f1b0481-2699-47d2-805b-219f76c29416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a1db92e28444854b9de817dd36104ea'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.579513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1279/2507] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:06 2022] DELETE /identity/v3/users/4a1db92e28444854b9de817dd36104ea => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:06.753423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:06.765544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3881b-a0e5-4b96-a3d2-2a18fb201a73 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:06.823559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:06.824197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:06.824483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:06.824764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] PATH_INFO: `/v3/projects/dcffc8bbe6ec425997d67634a0f7d724` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:06.826581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcffc8bbe6ec425997d67634a0f7d724)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:06.828478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:06.873929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcffc8bbe6ec425997d67634a0f7d724', 'request_id': 'req-6e5fdc20-2cbb-4c06-931a-81024de51066'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.881134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcffc8bbe6ec425997d67634a0f7d724', 'request_id': 'req-6e5fdc20-2cbb-4c06-931a-81024de51066'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.886781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcffc8bbe6ec425997d67634a0f7d724', 'request_id': 'req-6e5fdc20-2cbb-4c06-931a-81024de51066'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.898689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] The token cache is being invalidate because project dcffc8bbe6ec425997d67634a0f7d724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:06.898979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.899611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6e5fdc20-2cbb-4c06-931a-81024de51066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:06.901087 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1229/2508] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:06 2022] DELETE /identity/v3/projects/dcffc8bbe6ec425997d67634a0f7d724 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:06.906064 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2278-5d21-4d7a-ae4e-61ada33ec6ac None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:06.906358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2278-5d21-4d7a-ae4e-61ada33ec6ac None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:06.906635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2278-5d21-4d7a-ae4e-61ada33ec6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:06.965681 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d29d2278-5d21-4d7a-ae4e-61ada33ec6ac None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:06.966881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d29d2278-5d21-4d7a-ae4e-61ada33ec6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:07.026304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1280/2509] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:45:07.035064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b3a7d-1ed3-4c5a-a235-c45e18c03e0a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.035352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b3a7d-1ed3-4c5a-a235-c45e18c03e0a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.035639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b3a7d-1ed3-4c5a-a235-c45e18c03e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.055447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-428b3a7d-1ed3-4c5a-a235-c45e18c03e0a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:07.056643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-428b3a7d-1ed3-4c5a-a235-c45e18c03e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:07.065046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1230/2510] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:07.069556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d2278-5d21-4d7a-ae4e-61ada33ec6ac None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:07.079968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d2278-5d21-4d7a-ae4e-61ada33ec6ac None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:07.084903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:07.085546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.085829 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.086099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.086795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:07.087102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:07.088691 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:07.101567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1281/2511] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:45:07.105849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b3a7d-1ed3-4c5a-a235-c45e18c03e0a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:07.110927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:07.117282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b3a7d-1ed3-4c5a-a235-c45e18c03e0a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:07.123665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28879a-99c3-4b01-bc17-2d76f9bcb2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:07.124275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28879a-99c3-4b01-bc17-2d76f9bcb2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.124557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28879a-99c3-4b01-bc17-2d76f9bcb2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.124832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28879a-99c3-4b01-bc17-2d76f9bcb2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.125547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28879a-99c3-4b01-bc17-2d76f9bcb2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:07.127450 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28879a-99c3-4b01-bc17-2d76f9bcb2b6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:07.154256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1231/2512] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:07 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:45:07.158431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:07.164806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:07.168627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366c6a6-8309-418c-8de7-60947ec2708e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:07.232758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6956b-8f38-4354-aa50-4deb630bdff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:07.233415 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6956b-8f38-4354-aa50-4deb630bdff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.233697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6956b-8f38-4354-aa50-4deb630bdff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.233968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6956b-8f38-4354-aa50-4deb630bdff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.234639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6956b-8f38-4354-aa50-4deb630bdff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:07.236585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6956b-8f38-4354-aa50-4deb630bdff0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:07.261549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1282/2513] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:07 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:07.263324 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28879a-99c3-4b01-bc17-2d76f9bcb2b6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:07.273805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28879a-99c3-4b01-bc17-2d76f9bcb2b6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:07.279221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c0de3-c825-45c6-a1bb-af3aa5ec0654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:07.279837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0de3-c825-45c6-a1bb-af3aa5ec0654 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.280118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0de3-c825-45c6-a1bb-af3aa5ec0654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.280497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0de3-c825-45c6-a1bb-af3aa5ec0654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.281191 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c0de3-c825-45c6-a1bb-af3aa5ec0654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:07.282967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c0de3-c825-45c6-a1bb-af3aa5ec0654 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:07.293503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1232/2514] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:07.297871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6956b-8f38-4354-aa50-4deb630bdff0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:07.309175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6956b-8f38-4354-aa50-4deb630bdff0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:07.314080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357b22e-08dc-4a8f-a72b-e2608e203951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:07.314695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b22e-08dc-4a8f-a72b-e2608e203951 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.314976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b22e-08dc-4a8f-a72b-e2608e203951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.315239 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b22e-08dc-4a8f-a72b-e2608e203951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.315936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357b22e-08dc-4a8f-a72b-e2608e203951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:07.317831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357b22e-08dc-4a8f-a72b-e2608e203951 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:07.329974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1283/2515] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:07.334563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c0de3-c825-45c6-a1bb-af3aa5ec0654 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:07.346643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c0de3-c825-45c6-a1bb-af3aa5ec0654 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:07.351344 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:07.352000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.352283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.352556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] PATH_INFO: `/v3/projects/68344b41c4664159ad624e92e422d892/users/1cb6ce5989c843bd9cc8f9a45c87ae66/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.367511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] RBAC: Authorizing `identity:create_grant(project_id=68344b41c4664159ad624e92e422d892, user_id=1cb6ce5989c843bd9cc8f9a45c87ae66, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:07.370213 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:07.386211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1233/2516] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:07 2022] PUT /identity/v3/projects/68344b41c4664159ad624e92e422d892/users/1cb6ce5989c843bd9cc8f9a45c87ae66/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:45:07.398224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1e8aa-0bf9-4f63-874c-6988a8a7b50f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.398516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1e8aa-0bf9-4f63-874c-6988a8a7b50f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.398796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1e8aa-0bf9-4f63-874c-6988a8a7b50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.429611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8bf1e8aa-0bf9-4f63-874c-6988a8a7b50f None None] MFA Rules not processed for user `1cb6ce5989c843bd9cc8f9a45c87ae66`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:07.431205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8bf1e8aa-0bf9-4f63-874c-6988a8a7b50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:07.502604 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1284/2517] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:07.514660 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:07.516364 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:07.650038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1355f64-371b-40a8-a924-821b7d28522d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:07.657338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf7f3e-2161-43f6-859e-06680d47664e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:07.657966 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf7f3e-2161-43f6-859e-06680d47664e service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:07.658243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf7f3e-2161-43f6-859e-06680d47664e service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:07.658515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf7f3e-2161-43f6-859e-06680d47664e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:07.666765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf7f3e-2161-43f6-859e-06680d47664e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:07.668347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf7f3e-2161-43f6-859e-06680d47664e service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:07.678409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1234/2518] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:10.390198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1e8aa-0bf9-4f63-874c-6988a8a7b50f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:10.454541 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1e8aa-0bf9-4f63-874c-6988a8a7b50f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:10.459167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:10.459813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:10.460099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:10.460378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] PATH_INFO: `/v3/users/1cb6ce5989c843bd9cc8f9a45c87ae66` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:10.461576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cb6ce5989c843bd9cc8f9a45c87ae66)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:10.463485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:10.478134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:10.515165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cb6ce5989c843bd9cc8f9a45c87ae66'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:10.522016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cb6ce5989c843bd9cc8f9a45c87ae66'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:10.530325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cb6ce5989c843bd9cc8f9a45c87ae66'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:10.530920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cb6ce5989c843bd9cc8f9a45c87ae66'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:10.542874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cb6ce5989c843bd9cc8f9a45c87ae66'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:10.549509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1285/2519] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:10 2022] DELETE /identity/v3/users/1cb6ce5989c843bd9cc8f9a45c87ae66 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:10.553971 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee51bc-6910-45ae-ae5b-c33f77132bf3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:10.554466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee51bc-6910-45ae-ae5b-c33f77132bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:10.554584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee51bc-6910-45ae-ae5b-c33f77132bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:10.573038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b8ee51bc-6910-45ae-ae5b-c33f77132bf3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:10.574278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b8ee51bc-6910-45ae-ae5b-c33f77132bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:10.623944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:10.635928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1235/2520] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:10.646145 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:10.647549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:10.708108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b471d-b0c6-4c1e-afc1-cc0e8b85dd3e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:10.711773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbad669-faf0-4a44-997b-782d2797f2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:10.712386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbad669-faf0-4a44-997b-782d2797f2eb service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:10.712672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbad669-faf0-4a44-997b-782d2797f2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:10.712949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbad669-faf0-4a44-997b-782d2797f2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:10.722727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbad669-faf0-4a44-997b-782d2797f2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:10.724146 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbad669-faf0-4a44-997b-782d2797f2eb service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:10.734353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1286/2521] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:11.017780 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee51bc-6910-45ae-ae5b-c33f77132bf3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:11.019285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b8ee51bc-6910-45ae-ae5b-c33f77132bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:11.032007 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee51bc-6910-45ae-ae5b-c33f77132bf3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:11.036680 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:11.037355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.037643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.037918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] PATH_INFO: `/v3/projects/68344b41c4664159ad624e92e422d892` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.039083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68344b41c4664159ad624e92e422d892)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:11.040976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:11.088002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68344b41c4664159ad624e92e422d892', 'request_id': 'req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:11.095490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68344b41c4664159ad624e92e422d892', 'request_id': 'req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:11.101508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68344b41c4664159ad624e92e422d892', 'request_id': 'req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:11.112779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] The token cache is being invalidate because project 68344b41c4664159ad624e92e422d892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:11.113067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:11.113732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-ef33cd8f-b0ed-445a-921f-8fbc12b6c927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:11.115286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1236/2522] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:11 2022] DELETE /identity/v3/projects/68344b41c4664159ad624e92e422d892 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:11.120107 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7977d-fb0e-4dcc-a8c7-c3a4b36aa25a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.120394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7977d-fb0e-4dcc-a8c7-c3a4b36aa25a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.120673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7977d-fb0e-4dcc-a8c7-c3a4b36aa25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.141088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ada7977d-fb0e-4dcc-a8c7-c3a4b36aa25a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:11.142247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ada7977d-fb0e-4dcc-a8c7-c3a4b36aa25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:11.201901 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1287/2523] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:11.209748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384d908-4e30-4f47-bfbd-2a0b22f70465 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.210041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384d908-4e30-4f47-bfbd-2a0b22f70465 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.210320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384d908-4e30-4f47-bfbd-2a0b22f70465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.230198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-1384d908-4e30-4f47-bfbd-2a0b22f70465 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:11.231540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1384d908-4e30-4f47-bfbd-2a0b22f70465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:11.239460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1237/2524] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:11.243751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7977d-fb0e-4dcc-a8c7-c3a4b36aa25a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:11.254288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7977d-fb0e-4dcc-a8c7-c3a4b36aa25a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:11.264080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:11.264707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.265010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.265279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.266000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:11.266335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:11.267964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:11.284263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1288/2525] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:45:11.288973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384d908-4e30-4f47-bfbd-2a0b22f70465 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:11.298651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384d908-4e30-4f47-bfbd-2a0b22f70465 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:11.303244 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9361363-7114-4365-a26f-526098b4601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:11.303846 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361363-7114-4365-a26f-526098b4601e admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.304128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361363-7114-4365-a26f-526098b4601e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.304407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361363-7114-4365-a26f-526098b4601e admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.305065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9361363-7114-4365-a26f-526098b4601e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:11.307067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9361363-7114-4365-a26f-526098b4601e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:11.335857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1238/2526] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:11 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:11.340127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:11.351540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf5936-dc22-4534-8c78-adbdc4171b3e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:11.370090 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:11.411220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f841-1ca6-4b14-909c-b51870e21f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:11.411852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f841-1ca6-4b14-909c-b51870e21f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.412051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f841-1ca6-4b14-909c-b51870e21f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.412330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f841-1ca6-4b14-909c-b51870e21f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.413006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51f841-1ca6-4b14-909c-b51870e21f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:11.414974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51f841-1ca6-4b14-909c-b51870e21f5c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:11.439035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1289/2527] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:11 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:11.440210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9361363-7114-4365-a26f-526098b4601e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:11.451354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9361363-7114-4365-a26f-526098b4601e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:11.456143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224c5e-4be2-408e-a817-2db3db978053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:11.456664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224c5e-4be2-408e-a817-2db3db978053 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.456947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224c5e-4be2-408e-a817-2db3db978053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.457224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224c5e-4be2-408e-a817-2db3db978053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.457952 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05224c5e-4be2-408e-a817-2db3db978053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:11.459570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05224c5e-4be2-408e-a817-2db3db978053 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:11.469611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1239/2528] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:45:11.473649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f841-1ca6-4b14-909c-b51870e21f5c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:11.483673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f841-1ca6-4b14-909c-b51870e21f5c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:11.488490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc0dff-26bf-4de7-9aa1-1e29906e2274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:11.489128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0dff-26bf-4de7-9aa1-1e29906e2274 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.489418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0dff-26bf-4de7-9aa1-1e29906e2274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.489694 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0dff-26bf-4de7-9aa1-1e29906e2274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.490391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc0dff-26bf-4de7-9aa1-1e29906e2274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:11.492034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc0dff-26bf-4de7-9aa1-1e29906e2274 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:11.503462 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1290/2529] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:11.507546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224c5e-4be2-408e-a817-2db3db978053 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:11.518107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224c5e-4be2-408e-a817-2db3db978053 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:11.522650 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:11.523300 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.523586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.523861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] PATH_INFO: `/v3/projects/794dd8c613a74767b78b43f300930c73/users/dc4676fac09649e59e3f6d23796842f9/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.538970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=794dd8c613a74767b78b43f300930c73, user_id=dc4676fac09649e59e3f6d23796842f9, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:11.541617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:11.555341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1240/2530] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:11 2022] PUT /identity/v3/projects/794dd8c613a74767b78b43f300930c73/users/dc4676fac09649e59e3f6d23796842f9/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:11.566583 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fe2ec-f067-4067-a453-4aa8659e3034 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.566867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fe2ec-f067-4067-a453-4aa8659e3034 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.567137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fe2ec-f067-4067-a453-4aa8659e3034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.595975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-194fe2ec-f067-4067-a453-4aa8659e3034 None None] MFA Rules not processed for user `dc4676fac09649e59e3f6d23796842f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:11.597198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-194fe2ec-f067-4067-a453-4aa8659e3034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:11.663727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1291/2531] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:45:11.675172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:11.676883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:11.749215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d4c9c-9479-4e0f-a34c-a2ceaf71ab79 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:11.754708 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e56e3-383b-4732-a1d9-3a740f84512d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:11.755314 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e56e3-383b-4732-a1d9-3a740f84512d service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:11.755592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e56e3-383b-4732-a1d9-3a740f84512d service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:11.755862 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e56e3-383b-4732-a1d9-3a740f84512d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:11.763990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e56e3-383b-4732-a1d9-3a740f84512d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:11.765740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e56e3-383b-4732-a1d9-3a740f84512d service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:11.770164 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:11.775157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1241/2532] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:12.002305 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fe2ec-f067-4067-a453-4aa8659e3034 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:12.064509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fe2ec-f067-4067-a453-4aa8659e3034 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:12.069493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:12.070138 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.070426 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.070706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] PATH_INFO: `/v3/users/dc4676fac09649e59e3f6d23796842f9` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.071866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc4676fac09649e59e3f6d23796842f9)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:12.073938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:12.110480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:12.124656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc4676fac09649e59e3f6d23796842f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.131551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc4676fac09649e59e3f6d23796842f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.139392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc4676fac09649e59e3f6d23796842f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.140093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc4676fac09649e59e3f6d23796842f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.151821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc4676fac09649e59e3f6d23796842f9'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.157715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1292/2533] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:12 2022] DELETE /identity/v3/users/dc4676fac09649e59e3f6d23796842f9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:12.162177 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b4e19-617e-4dfc-98b3-e91d50906919 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.162472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b4e19-617e-4dfc-98b3-e91d50906919 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.162762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b4e19-617e-4dfc-98b3-e91d50906919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.181054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c20b4e19-617e-4dfc-98b3-e91d50906919 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:12.182230 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c20b4e19-617e-4dfc-98b3-e91d50906919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:12.240993 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1242/2534] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:12.251120 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:12.252528 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:12.311576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa163f-e097-49e6-9c84-33ea6d567562 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:12.315341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe8aff-f56b-4045-92c2-3f1267ee066b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:12.315958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe8aff-f56b-4045-92c2-3f1267ee066b service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.316492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe8aff-f56b-4045-92c2-3f1267ee066b service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.316765 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe8aff-f56b-4045-92c2-3f1267ee066b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.326505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe8aff-f56b-4045-92c2-3f1267ee066b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:12.327933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe8aff-f56b-4045-92c2-3f1267ee066b service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:12.337405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1293/2535] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:12.608697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b4e19-617e-4dfc-98b3-e91d50906919 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:12.610263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c20b4e19-617e-4dfc-98b3-e91d50906919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:12.623356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b4e19-617e-4dfc-98b3-e91d50906919 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:12.628090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:12.628813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.629166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.629537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] PATH_INFO: `/v3/projects/794dd8c613a74767b78b43f300930c73` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.630785 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=794dd8c613a74767b78b43f300930c73)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:12.632987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:12.680806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794dd8c613a74767b78b43f300930c73', 'request_id': 'req-c320179c-8e90-499c-8b3d-26b2a80376f5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.689328 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794dd8c613a74767b78b43f300930c73', 'request_id': 'req-c320179c-8e90-499c-8b3d-26b2a80376f5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.695329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794dd8c613a74767b78b43f300930c73', 'request_id': 'req-c320179c-8e90-499c-8b3d-26b2a80376f5'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.706362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] The token cache is being invalidate because project 794dd8c613a74767b78b43f300930c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:12.706652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.707295 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c320179c-8e90-499c-8b3d-26b2a80376f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:12.708794 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1243/2536] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:12 2022] DELETE /identity/v3/projects/794dd8c613a74767b78b43f300930c73 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:12.713737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f11a-3061-41a8-a95f-42a3514143c1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.714208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f11a-3061-41a8-a95f-42a3514143c1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.714349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f11a-3061-41a8-a95f-42a3514143c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.733919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c106f11a-3061-41a8-a95f-42a3514143c1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:12.735149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c106f11a-3061-41a8-a95f-42a3514143c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:12.793240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1294/2537] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:12.803679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630e15-128c-448d-b936-3b573f67b050 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.803972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630e15-128c-448d-b936-3b573f67b050 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.804275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630e15-128c-448d-b936-3b573f67b050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.823856 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-14630e15-128c-448d-b936-3b573f67b050 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:12.825173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-14630e15-128c-448d-b936-3b573f67b050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:12.832986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1244/2538] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:12.837117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106f11a-3061-41a8-a95f-42a3514143c1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:12.847628 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106f11a-3061-41a8-a95f-42a3514143c1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:12.852207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:12.852815 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.853099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.853405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.854102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:12.854408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:12.856010 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:12.868077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1295/2539] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:45:12.872242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14630e15-128c-448d-b936-3b573f67b050 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:12.881595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14630e15-128c-448d-b936-3b573f67b050 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:12.886297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395190d9-687d-4cf5-bad5-18b541f59327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:12.886899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395190d9-687d-4cf5-bad5-18b541f59327 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.887183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395190d9-687d-4cf5-bad5-18b541f59327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.887461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395190d9-687d-4cf5-bad5-18b541f59327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.888126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395190d9-687d-4cf5-bad5-18b541f59327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:12.890054 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395190d9-687d-4cf5-bad5-18b541f59327 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:12.917070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1245/2540] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:12 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:12.922132 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:12.932019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbabc3-0766-4c68-b2ae-796314922c05 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:12.970485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:12.990411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec90f0-6ea0-4cfa-9b13-64a5788684dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:12.991022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec90f0-6ea0-4cfa-9b13-64a5788684dd admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:12.991300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec90f0-6ea0-4cfa-9b13-64a5788684dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:12.991569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec90f0-6ea0-4cfa-9b13-64a5788684dd admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:12.992256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec90f0-6ea0-4cfa-9b13-64a5788684dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:12.994178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec90f0-6ea0-4cfa-9b13-64a5788684dd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.017830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1296/2541] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:12 2022] POST /identity/v3/users => generated 278 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:13.022115 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395190d9-687d-4cf5-bad5-18b541f59327 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.033211 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395190d9-687d-4cf5-bad5-18b541f59327 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.037861 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc6edc-5f05-4a1b-974b-d5d105290dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.038592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc6edc-5f05-4a1b-974b-d5d105290dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.038945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc6edc-5f05-4a1b-974b-d5d105290dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.039356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc6edc-5f05-4a1b-974b-d5d105290dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.040120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc6edc-5f05-4a1b-974b-d5d105290dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.041795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc6edc-5f05-4a1b-974b-d5d105290dd6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.051897 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1246/2542] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:13.055860 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec90f0-6ea0-4cfa-9b13-64a5788684dd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.066858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec90f0-6ea0-4cfa-9b13-64a5788684dd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.071510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb499266-0dda-4fab-a39d-6b2f50686164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.072118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb499266-0dda-4fab-a39d-6b2f50686164 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.072403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb499266-0dda-4fab-a39d-6b2f50686164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.072679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb499266-0dda-4fab-a39d-6b2f50686164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.073407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb499266-0dda-4fab-a39d-6b2f50686164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.075596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb499266-0dda-4fab-a39d-6b2f50686164 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.085697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1297/2543] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:13.089781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc6edc-5f05-4a1b-974b-d5d105290dd6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.099560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc6edc-5f05-4a1b-974b-d5d105290dd6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.104917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551aebd-e7cd-455f-bbb0-350a33b835cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.105618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551aebd-e7cd-455f-bbb0-350a33b835cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.105944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551aebd-e7cd-455f-bbb0-350a33b835cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.106762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551aebd-e7cd-455f-bbb0-350a33b835cb admin admin] PATH_INFO: `/v3/projects/78f5db7c5bbc42c4b339568e7960d36b/users/dcfcad0a497045fb90e56b55b7b14ea4/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.121127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551aebd-e7cd-455f-bbb0-350a33b835cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f5db7c5bbc42c4b339568e7960d36b, user_id=dcfcad0a497045fb90e56b55b7b14ea4, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.124351 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551aebd-e7cd-455f-bbb0-350a33b835cb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.136503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1247/2544] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:13 2022] PUT /identity/v3/projects/78f5db7c5bbc42c4b339568e7960d36b/users/dcfcad0a497045fb90e56b55b7b14ea4/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:45:13.147240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47aa1b-fa3b-47a1-83aa-fc8fa7a36124 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.147540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47aa1b-fa3b-47a1-83aa-fc8fa7a36124 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.147822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47aa1b-fa3b-47a1-83aa-fc8fa7a36124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.175964 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6c47aa1b-fa3b-47a1-83aa-fc8fa7a36124 None None] MFA Rules not processed for user `dcfcad0a497045fb90e56b55b7b14ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:13.177163 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6c47aa1b-fa3b-47a1-83aa-fc8fa7a36124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:13.241512 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1298/2545] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:13 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:13.246096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551aebd-e7cd-455f-bbb0-350a33b835cb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.268498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:13.311594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551aebd-e7cd-455f-bbb0-350a33b835cb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.316429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16c07f-2afb-4aa9-9e9d-b680db26a36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.317039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16c07f-2afb-4aa9-9e9d-b680db26a36f admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.317341 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16c07f-2afb-4aa9-9e9d-b680db26a36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.317613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16c07f-2afb-4aa9-9e9d-b680db26a36f admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.318306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16c07f-2afb-4aa9-9e9d-b680db26a36f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.320197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16c07f-2afb-4aa9-9e9d-b680db26a36f admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.346831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1248/2546] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:13 2022] POST /identity/v3/projects => generated 357 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:13.350903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47aa1b-fa3b-47a1-83aa-fc8fa7a36124 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.361587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47aa1b-fa3b-47a1-83aa-fc8fa7a36124 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.416877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572a12c-cfdf-411f-8900-4502a7672b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.417521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a12c-cfdf-411f-8900-4502a7672b56 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.417806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a12c-cfdf-411f-8900-4502a7672b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.418083 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a12c-cfdf-411f-8900-4502a7672b56 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.418913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572a12c-cfdf-411f-8900-4502a7672b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.420836 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572a12c-cfdf-411f-8900-4502a7672b56 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.442346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1299/2547] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:13 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:13.446321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16c07f-2afb-4aa9-9e9d-b680db26a36f admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.457746 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16c07f-2afb-4aa9-9e9d-b680db26a36f admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.462389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675e7cc-6421-41c9-b692-f901032849d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.463000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675e7cc-6421-41c9-b692-f901032849d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.463283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675e7cc-6421-41c9-b692-f901032849d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.463559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675e7cc-6421-41c9-b692-f901032849d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.464256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675e7cc-6421-41c9-b692-f901032849d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.466500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675e7cc-6421-41c9-b692-f901032849d5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.477470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1249/2548] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:13.481779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572a12c-cfdf-411f-8900-4502a7672b56 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.492106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572a12c-cfdf-411f-8900-4502a7672b56 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.497213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16ce05-94ee-4683-8bfd-7e1fdbbf6ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.497907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16ce05-94ee-4683-8bfd-7e1fdbbf6ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.498207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16ce05-94ee-4683-8bfd-7e1fdbbf6ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.498466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16ce05-94ee-4683-8bfd-7e1fdbbf6ee8 admin admin] PATH_INFO: `/v3/projects/8c29a4c056b348e286037ccca0e4a216/users/0eea01ad6f7d4585ae72da9321c325f4/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.513030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16ce05-94ee-4683-8bfd-7e1fdbbf6ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c29a4c056b348e286037ccca0e4a216, user_id=0eea01ad6f7d4585ae72da9321c325f4, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.515710 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16ce05-94ee-4683-8bfd-7e1fdbbf6ee8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.530117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1300/2549] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:13 2022] PUT /identity/v3/projects/8c29a4c056b348e286037ccca0e4a216/users/0eea01ad6f7d4585ae72da9321c325f4/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:13.534161 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675e7cc-6421-41c9-b692-f901032849d5 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.544432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675e7cc-6421-41c9-b692-f901032849d5 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.599748 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948decf7-516d-4ecc-a066-94e221e05b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.600400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948decf7-516d-4ecc-a066-94e221e05b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.600700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948decf7-516d-4ecc-a066-94e221e05b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.600973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948decf7-516d-4ecc-a066-94e221e05b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.601706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948decf7-516d-4ecc-a066-94e221e05b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.603285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948decf7-516d-4ecc-a066-94e221e05b2c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.613509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1250/2550] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:13.617793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16ce05-94ee-4683-8bfd-7e1fdbbf6ee8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.627984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16ce05-94ee-4683-8bfd-7e1fdbbf6ee8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.632503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.633151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.633469 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.633741 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] PATH_INFO: `/v3/projects/8c29a4c056b348e286037ccca0e4a216/users/0eea01ad6f7d4585ae72da9321c325f4/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.635651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c29a4c056b348e286037ccca0e4a216, user_id=0eea01ad6f7d4585ae72da9321c325f4, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.638285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.644281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:13.649103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1301/2551] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:13 2022] PUT /identity/v3/projects/8c29a4c056b348e286037ccca0e4a216/users/0eea01ad6f7d4585ae72da9321c325f4/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:13.660059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac48351-76f1-42b4-832e-9be60175b9f9 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.660352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac48351-76f1-42b4-832e-9be60175b9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.660633 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac48351-76f1-42b4-832e-9be60175b9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.688771 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fac48351-76f1-42b4-832e-9be60175b9f9 None None] MFA Rules not processed for user `0eea01ad6f7d4585ae72da9321c325f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:13.690036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fac48351-76f1-42b4-832e-9be60175b9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:13.759761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1251/2552] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:13 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:13.771593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.773264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:13.831184 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9b9ef-01d1-498b-b213-ce1856390a87 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.834743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57e5d0-a7b6-4941-a00b-bc64d2259f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.835351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57e5d0-a7b6-4941-a00b-bc64d2259f7f service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.835627 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57e5d0-a7b6-4941-a00b-bc64d2259f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.835898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57e5d0-a7b6-4941-a00b-bc64d2259f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.845203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57e5d0-a7b6-4941-a00b-bc64d2259f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:13.846652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57e5d0-a7b6-4941-a00b-bc64d2259f7f service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:13.856074 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1302/2553] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:13 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:13.934490 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac48351-76f1-42b4-832e-9be60175b9f9 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:13.944155 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac48351-76f1-42b4-832e-9be60175b9f9 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:13.947701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43043ec7-b5be-40fa-8f37-fd9d5ff0dbe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:13.948302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43043ec7-b5be-40fa-8f37-fd9d5ff0dbe8 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:13.948581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43043ec7-b5be-40fa-8f37-fd9d5ff0dbe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:13.948853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43043ec7-b5be-40fa-8f37-fd9d5ff0dbe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:13.960977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:14.011745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43043ec7-b5be-40fa-8f37-fd9d5ff0dbe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:14.013335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43043ec7-b5be-40fa-8f37-fd9d5ff0dbe8 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:14.023091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1252/2554] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:13 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:20.071665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57e5d0-a7b6-4941-a00b-bc64d2259f7f service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:20.080488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57e5d0-a7b6-4941-a00b-bc64d2259f7f service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:20.084072 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:20.084678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:20.084960 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:20.085234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:20.087057 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:20.146213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:20.147618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:20.156703 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1303/2555] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:24.019132 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=21 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:28.128740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a905c-0232-4a97-ba23-6557e127b294 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:28.129381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a905c-0232-4a97-ba23-6557e127b294 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:28.129381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a905c-0232-4a97-ba23-6557e127b294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:28.148239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-fc6a905c-0232-4a97-ba23-6557e127b294 None None] MFA Rules not processed for user `13ca4be686f84c279f87016c20129833`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:28.149480 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fc6a905c-0232-4a97-ba23-6557e127b294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:28.156582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1253/2556] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:28.493457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:28.494925 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:28.584462 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45749db6-0ae2-4a8d-86fc-b8de7da707d5 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:28.588265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18f8fe-465a-47d6-abde-8bb2c1ba99b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:28.588998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18f8fe-465a-47d6-abde-8bb2c1ba99b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:28.589343 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18f8fe-465a-47d6-abde-8bb2c1ba99b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:28.589679 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18f8fe-465a-47d6-abde-8bb2c1ba99b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:28.591493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ff18f8fe-465a-47d6-abde-8bb2c1ba99b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:28.602948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18f8fe-465a-47d6-abde-8bb2c1ba99b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:28.604400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18f8fe-465a-47d6-abde-8bb2c1ba99b5 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:28.613639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1304/2557] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:29.965827 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a905c-0232-4a97-ba23-6557e127b294 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:30.031158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a905c-0232-4a97-ba23-6557e127b294 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:30.041582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:30.041582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:30.041582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:30.041582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] PATH_INFO: `/v3/users/dcfcad0a497045fb90e56b55b7b14ea4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:30.042002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcfcad0a497045fb90e56b55b7b14ea4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:30.042002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:30.097245 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcfcad0a497045fb90e56b55b7b14ea4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.106267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcfcad0a497045fb90e56b55b7b14ea4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.107027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcfcad0a497045fb90e56b55b7b14ea4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.136943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcfcad0a497045fb90e56b55b7b14ea4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.137772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcfcad0a497045fb90e56b55b7b14ea4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.151597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1254/2558] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:29 2022] DELETE /identity/v3/users/dcfcad0a497045fb90e56b55b7b14ea4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:30.156334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac4182-8c0a-4b68-a1c5-c49d26f9ddd0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:30.156822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac4182-8c0a-4b68-a1c5-c49d26f9ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:30.156928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac4182-8c0a-4b68-a1c5-c49d26f9ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:30.176276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-43ac4182-8c0a-4b68-a1c5-c49d26f9ddd0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:30.177519 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-43ac4182-8c0a-4b68-a1c5-c49d26f9ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:30.198954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:30.241706 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1305/2559] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:30.252354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:30.253942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:30.318881 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec6c60-45f4-4166-a103-84d875418758 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:30.322864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e730caa-d801-44bb-b701-dd8472ae04ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:30.323485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e730caa-d801-44bb-b701-dd8472ae04ab service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:30.323765 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e730caa-d801-44bb-b701-dd8472ae04ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:30.324045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e730caa-d801-44bb-b701-dd8472ae04ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:30.334168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e730caa-d801-44bb-b701-dd8472ae04ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:30.335624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e730caa-d801-44bb-b701-dd8472ae04ab service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:30.346222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1255/2560] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:30.637510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac4182-8c0a-4b68-a1c5-c49d26f9ddd0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:30.638939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-43ac4182-8c0a-4b68-a1c5-c49d26f9ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:30.659717 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac4182-8c0a-4b68-a1c5-c49d26f9ddd0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:30.665252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:30.665936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:30.666347 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:30.666633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] PATH_INFO: `/v3/projects/78f5db7c5bbc42c4b339568e7960d36b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:30.667874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f5db7c5bbc42c4b339568e7960d36b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:30.670451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:30.718723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f5db7c5bbc42c4b339568e7960d36b', 'request_id': 'req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.724372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f5db7c5bbc42c4b339568e7960d36b', 'request_id': 'req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.731209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f5db7c5bbc42c4b339568e7960d36b', 'request_id': 'req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.742127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] The token cache is being invalidate because project 78f5db7c5bbc42c4b339568e7960d36b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:30.742412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.743039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.744511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1306/2561] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:30 2022] DELETE /identity/v3/projects/78f5db7c5bbc42c4b339568e7960d36b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:30.748457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e730caa-d801-44bb-b701-dd8472ae04ab service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:30.759183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e730caa-d801-44bb-b701-dd8472ae04ab service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:30.772631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:30.819812 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:30.820385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:30.820670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:30.820949 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] PATH_INFO: `/v3/users/0eea01ad6f7d4585ae72da9321c325f4` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:30.822142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eea01ad6f7d4585ae72da9321c325f4)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:30.824104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:30.870310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eea01ad6f7d4585ae72da9321c325f4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.878638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eea01ad6f7d4585ae72da9321c325f4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.879256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eea01ad6f7d4585ae72da9321c325f4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.892892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eea01ad6f7d4585ae72da9321c325f4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.893608 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eea01ad6f7d4585ae72da9321c325f4'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:30.906070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1256/2562] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:30 2022] DELETE /identity/v3/users/0eea01ad6f7d4585ae72da9321c325f4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:31.152755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:31.154296 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:31.201227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:31.203930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:31.284936 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9daa80-5979-4bfd-8f95-ee6f1b9b4f08 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:31.292986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecf62e-7787-489b-aeb2-3ead05d2ee01 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:31.298410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:31.299053 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.299332 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.299603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] PATH_INFO: `/v3/projects/8c29a4c056b348e286037ccca0e4a216` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.300808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c29a4c056b348e286037ccca0e4a216)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:31.302849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:31.320121 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:31.343522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:31.353802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c29a4c056b348e286037ccca0e4a216', 'request_id': 'req-91c1cb3f-acf3-493d-a7f1-8973ce93c990'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:31.357278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f8e18-20ff-490e-aafc-3b22233af0ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:31.357975 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f8e18-20ff-490e-aafc-3b22233af0ed service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.358253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f8e18-20ff-490e-aafc-3b22233af0ed service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.358525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f8e18-20ff-490e-aafc-3b22233af0ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.360588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-000f8e18-20ff-490e-aafc-3b22233af0ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:31.362061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c29a4c056b348e286037ccca0e4a216', 'request_id': 'req-91c1cb3f-acf3-493d-a7f1-8973ce93c990'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:31.368225 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c29a4c056b348e286037ccca0e4a216', 'request_id': 'req-91c1cb3f-acf3-493d-a7f1-8973ce93c990'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:31.371875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f8e18-20ff-490e-aafc-3b22233af0ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:31.373537 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f8e18-20ff-490e-aafc-3b22233af0ed service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:31.380060 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] The token cache is being invalidate because project 8c29a4c056b348e286037ccca0e4a216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:31.380352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:31.380990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-91c1cb3f-acf3-493d-a7f1-8973ce93c990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:31.382523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1257/2563] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:31 2022] DELETE /identity/v3/projects/8c29a4c056b348e286037ccca0e4a216 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:45:31.383696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1307/2564] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:31.387093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7bee-41b1-421e-9a12-0d3f1673150b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.387384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7bee-41b1-421e-9a12-0d3f1673150b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.387668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7bee-41b1-421e-9a12-0d3f1673150b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.408867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-a34f7bee-41b1-421e-9a12-0d3f1673150b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:31.410126 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a34f7bee-41b1-421e-9a12-0d3f1673150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:31.485247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1258/2565] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:31.492994 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f840e3-e70b-4e5b-9c13-79951f6f714e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.493285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f840e3-e70b-4e5b-9c13-79951f6f714e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.493602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f840e3-e70b-4e5b-9c13-79951f6f714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.514193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-87f840e3-e70b-4e5b-9c13-79951f6f714e None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:31.515986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-87f840e3-e70b-4e5b-9c13-79951f6f714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:31.523757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1308/2566] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:31.528009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f7bee-41b1-421e-9a12-0d3f1673150b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:31.537920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f7bee-41b1-421e-9a12-0d3f1673150b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:31.542565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:31.543172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.543460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.543752 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.544424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:31.544731 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:31.546409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:31.558111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1259/2567] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:45:31.563457 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f840e3-e70b-4e5b-9c13-79951f6f714e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:31.574818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f840e3-e70b-4e5b-9c13-79951f6f714e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:31.580211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586aad5-07e3-4774-8b45-c6930ebb6c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:31.581330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586aad5-07e3-4774-8b45-c6930ebb6c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.581746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586aad5-07e3-4774-8b45-c6930ebb6c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.581872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586aad5-07e3-4774-8b45-c6930ebb6c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.582602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586aad5-07e3-4774-8b45-c6930ebb6c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:31.584683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586aad5-07e3-4774-8b45-c6930ebb6c51 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:31.620230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1309/2568] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:31 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:31.625174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:31.639016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f282bbb-f2b9-4153-a87d-803fb510b84a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:31.726697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a8c02-e4d8-45ac-b692-f6c0b8ad1952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:31.727990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a8c02-e4d8-45ac-b692-f6c0b8ad1952 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.728275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a8c02-e4d8-45ac-b692-f6c0b8ad1952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.728551 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a8c02-e4d8-45ac-b692-f6c0b8ad1952 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.729221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a8c02-e4d8-45ac-b692-f6c0b8ad1952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:31.732423 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a8c02-e4d8-45ac-b692-f6c0b8ad1952 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:31.760821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1260/2569] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:31 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:31.765700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586aad5-07e3-4774-8b45-c6930ebb6c51 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:31.776614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586aad5-07e3-4774-8b45-c6930ebb6c51 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:31.782109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86eaa9-c6b6-4a44-a752-47bd823bfcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:31.782737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86eaa9-c6b6-4a44-a752-47bd823bfcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.783019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86eaa9-c6b6-4a44-a752-47bd823bfcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.783293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86eaa9-c6b6-4a44-a752-47bd823bfcf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.783987 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86eaa9-c6b6-4a44-a752-47bd823bfcf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:31.785636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86eaa9-c6b6-4a44-a752-47bd823bfcf5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:31.817336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1310/2570] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:31.820652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a8c02-e4d8-45ac-b692-f6c0b8ad1952 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:31.832088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a8c02-e4d8-45ac-b692-f6c0b8ad1952 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:31.836812 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfb0c6-380c-46e5-86d2-ce7e9827104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:31.837733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfb0c6-380c-46e5-86d2-ce7e9827104c admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.838033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfb0c6-380c-46e5-86d2-ce7e9827104c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.838322 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfb0c6-380c-46e5-86d2-ce7e9827104c admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.838973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdfb0c6-380c-46e5-86d2-ce7e9827104c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:31.840587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdfb0c6-380c-46e5-86d2-ce7e9827104c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:31.852392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1261/2571] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:31.856607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86eaa9-c6b6-4a44-a752-47bd823bfcf5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:31.867979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86eaa9-c6b6-4a44-a752-47bd823bfcf5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:31.872579 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:31.873241 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.874713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.875004 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] PATH_INFO: `/v3/projects/64fd9880c09c4abb92a81d8812208690/users/b3db2112b45a4230afc34cb8736bb8b7/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.890550 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64fd9880c09c4abb92a81d8812208690, user_id=b3db2112b45a4230afc34cb8736bb8b7, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:31.893238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:31.910958 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1311/2572] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:31 2022] PUT /identity/v3/projects/64fd9880c09c4abb92a81d8812208690/users/b3db2112b45a4230afc34cb8736bb8b7/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:31.922313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910004e-80fb-4057-a0f4-a243ee901706 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:31.922609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910004e-80fb-4057-a0f4-a243ee901706 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:31.922889 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910004e-80fb-4057-a0f4-a243ee901706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:31.952357 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5910004e-80fb-4057-a0f4-a243ee901706 None None] MFA Rules not processed for user `b3db2112b45a4230afc34cb8736bb8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:31.953982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5910004e-80fb-4057-a0f4-a243ee901706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:31.972350 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:32.023019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1262/2573] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:45:32.034932 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:32.036876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:32.125690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96c057-360e-4733-b25d-f7e68985a72e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:32.131806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce1e38-4441-4c32-84d7-562629afc674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:32.132442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce1e38-4441-4c32-84d7-562629afc674 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:32.132759 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce1e38-4441-4c32-84d7-562629afc674 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:32.133034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce1e38-4441-4c32-84d7-562629afc674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:32.143840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce1e38-4441-4c32-84d7-562629afc674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:32.145342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce1e38-4441-4c32-84d7-562629afc674 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:32.156492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1312/2574] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:32.179953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910004e-80fb-4057-a0f4-a243ee901706 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:32.259239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910004e-80fb-4057-a0f4-a243ee901706 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:32.265130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:32.266623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:32.266983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:32.267446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] PATH_INFO: `/v3/users/b3db2112b45a4230afc34cb8736bb8b7` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:32.268761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3db2112b45a4230afc34cb8736bb8b7)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:32.270813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:32.356286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3db2112b45a4230afc34cb8736bb8b7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:32.375606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3db2112b45a4230afc34cb8736bb8b7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:32.377253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3db2112b45a4230afc34cb8736bb8b7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:32.406497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3db2112b45a4230afc34cb8736bb8b7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:32.407264 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3db2112b45a4230afc34cb8736bb8b7'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:32.440435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1263/2575] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:32 2022] DELETE /identity/v3/users/b3db2112b45a4230afc34cb8736bb8b7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:32.442283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5f2e4-dd5c-40a1-80dc-34df28467ee9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:32.442772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5f2e4-dd5c-40a1-80dc-34df28467ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:32.442984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5f2e4-dd5c-40a1-80dc-34df28467ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:32.470998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-feb5f2e4-dd5c-40a1-80dc-34df28467ee9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:32.472233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-feb5f2e4-dd5c-40a1-80dc-34df28467ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:32.510423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:32.540698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1313/2576] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:32.551885 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:32.553319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:32.610800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:32.626354 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961595d5-cc38-4c5f-8f99-5aba726f4e05 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:32.630048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df9479-f0f2-4bc5-b1f2-3d3e5b15b942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:32.630614 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9479-f0f2-4bc5-b1f2-3d3e5b15b942 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:32.630898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9479-f0f2-4bc5-b1f2-3d3e5b15b942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:32.631174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9479-f0f2-4bc5-b1f2-3d3e5b15b942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:32.640974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df9479-f0f2-4bc5-b1f2-3d3e5b15b942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:32.642430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df9479-f0f2-4bc5-b1f2-3d3e5b15b942 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:32.652500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1264/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:33.156451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5f2e4-dd5c-40a1-80dc-34df28467ee9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:33.159446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-feb5f2e4-dd5c-40a1-80dc-34df28467ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:33.172536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5f2e4-dd5c-40a1-80dc-34df28467ee9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:33.178385 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:33.179141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.179498 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.179781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] PATH_INFO: `/v3/projects/64fd9880c09c4abb92a81d8812208690` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.181035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64fd9880c09c4abb92a81d8812208690)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:33.183186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:33.258029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64fd9880c09c4abb92a81d8812208690', 'request_id': 'req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:33.269496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64fd9880c09c4abb92a81d8812208690', 'request_id': 'req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:33.279350 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64fd9880c09c4abb92a81d8812208690', 'request_id': 'req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:33.293257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] The token cache is being invalidate because project 64fd9880c09c4abb92a81d8812208690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:33.293858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:33.294542 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a082c74f-f63e-441b-b3e3-6ca15e3e9f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:33.296830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1314/2578] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:33 2022] DELETE /identity/v3/projects/64fd9880c09c4abb92a81d8812208690 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:33.302912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a652f1e-4497-4e9f-b36a-66b908667fe7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.303401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a652f1e-4497-4e9f-b36a-66b908667fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.303497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a652f1e-4497-4e9f-b36a-66b908667fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.324593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2a652f1e-4497-4e9f-b36a-66b908667fe7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:33.326185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2a652f1e-4497-4e9f-b36a-66b908667fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:33.430792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1265/2579] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:33.438885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a2b36-4734-44ed-a18d-aa2e8e1607b1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.440037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a2b36-4734-44ed-a18d-aa2e8e1607b1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.440337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a2b36-4734-44ed-a18d-aa2e8e1607b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.503046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3b7a2b36-4734-44ed-a18d-aa2e8e1607b1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:33.504929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3b7a2b36-4734-44ed-a18d-aa2e8e1607b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:33.536547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1315/2580] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:33.540547 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a652f1e-4497-4e9f-b36a-66b908667fe7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:33.554705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a652f1e-4497-4e9f-b36a-66b908667fe7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:33.569159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:33.569970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.571302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.571671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.573378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:33.574413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:33.579950 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:33.619645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1266/2581] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:45:33.625735 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a2b36-4734-44ed-a18d-aa2e8e1607b1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:33.640952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a2b36-4734-44ed-a18d-aa2e8e1607b1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:33.655800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5c9fd-af5f-4463-9c7f-e14eb87eb10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:33.656699 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5c9fd-af5f-4463-9c7f-e14eb87eb10c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.657159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5c9fd-af5f-4463-9c7f-e14eb87eb10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.657608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5c9fd-af5f-4463-9c7f-e14eb87eb10c admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.658634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5c9fd-af5f-4463-9c7f-e14eb87eb10c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:33.660721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5c9fd-af5f-4463-9c7f-e14eb87eb10c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:33.736049 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1316/2582] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:33 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:33.737502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:33.752193 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341490d7-7ba6-4128-b329-9268b7bf2329 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:33.817474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea85eb0-5721-4d90-8716-9e8b1c4b1873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:33.818107 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea85eb0-5721-4d90-8716-9e8b1c4b1873 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.818390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea85eb0-5721-4d90-8716-9e8b1c4b1873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.819083 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea85eb0-5721-4d90-8716-9e8b1c4b1873 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.819335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea85eb0-5721-4d90-8716-9e8b1c4b1873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:33.821392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea85eb0-5721-4d90-8716-9e8b1c4b1873 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:33.850127 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1267/2583] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:33 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:33.851642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5c9fd-af5f-4463-9c7f-e14eb87eb10c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:33.863389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5c9fd-af5f-4463-9c7f-e14eb87eb10c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:33.868129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2721b90-c265-412b-85f4-05f222e6ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:33.868760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2721b90-c265-412b-85f4-05f222e6ee9f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.869043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2721b90-c265-412b-85f4-05f222e6ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.869340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2721b90-c265-412b-85f4-05f222e6ee9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.870054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2721b90-c265-412b-85f4-05f222e6ee9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:33.871647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2721b90-c265-412b-85f4-05f222e6ee9f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:33.889160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1317/2584] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:33.891320 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea85eb0-5721-4d90-8716-9e8b1c4b1873 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:33.902624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea85eb0-5721-4d90-8716-9e8b1c4b1873 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:33.908480 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a958e6b-279a-4b48-bfb2-4a3f25e67168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:33.909108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958e6b-279a-4b48-bfb2-4a3f25e67168 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.909444 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958e6b-279a-4b48-bfb2-4a3f25e67168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.909721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958e6b-279a-4b48-bfb2-4a3f25e67168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.910809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a958e6b-279a-4b48-bfb2-4a3f25e67168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:33.912339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a958e6b-279a-4b48-bfb2-4a3f25e67168 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:33.937723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1268/2585] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:33.939237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2721b90-c265-412b-85f4-05f222e6ee9f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:33.949663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2721b90-c265-412b-85f4-05f222e6ee9f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:33.956529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:33.957205 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:33.957716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:33.957992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] PATH_INFO: `/v3/projects/21571203026444339b747739c5bed02a/users/efd0eb7116e04b76ac5f522fca889f87/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:33.975113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21571203026444339b747739c5bed02a, user_id=efd0eb7116e04b76ac5f522fca889f87, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:33.978868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:33.997144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1318/2586] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:33 2022] PUT /identity/v3/projects/21571203026444339b747739c5bed02a/users/efd0eb7116e04b76ac5f522fca889f87/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:34.008359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d9a94-5478-4403-b12c-c5f9726afb2d None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:34.008359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d9a94-5478-4403-b12c-c5f9726afb2d None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:34.008605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d9a94-5478-4403-b12c-c5f9726afb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:34.046984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-5d6d9a94-5478-4403-b12c-c5f9726afb2d None None] MFA Rules not processed for user `efd0eb7116e04b76ac5f522fca889f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:34.048303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5d6d9a94-5478-4403-b12c-c5f9726afb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:34.090496 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:34.125736 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1269/2587] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:34.133713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:34.135587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:34.210201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc8c36-cc16-4f32-a981-64211db4df27 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:34.214442 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:34.215337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:34.215474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:34.215827 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:34.228398 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:34.229981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:34.244387 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1319/2588] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:39.948017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d9a94-5478-4403-b12c-c5f9726afb2d None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:39.949429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5d6d9a94-5478-4403-b12c-c5f9726afb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:40.006986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d9a94-5478-4403-b12c-c5f9726afb2d None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:40.010633 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:40.011256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:40.011541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:40.011820 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:40.013670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:40.080286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:40.081762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:40.092526 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1270/2589] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:43.528647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:43.530322 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:43.539634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:43.546364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f060-cd31-4cca-9622-666e816cb440 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:43.551660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:43.552587 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:43.553134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:43.553767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] PATH_INFO: `/v3/users/cf13ae714ad04d6696c273603bba9fbc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:43.555427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf13ae714ad04d6696c273603bba9fbc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:43.557726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:43.610849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf13ae714ad04d6696c273603bba9fbc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:43.632154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf13ae714ad04d6696c273603bba9fbc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:43.640872 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf13ae714ad04d6696c273603bba9fbc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:43.641543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf13ae714ad04d6696c273603bba9fbc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:43.654752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2f91718f-3329-4c0f-a8f2-abdbeefb2d45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf13ae714ad04d6696c273603bba9fbc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:43.661171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1320/2590] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:43 2022] DELETE /identity/v3/users/cf13ae714ad04d6696c273603bba9fbc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:43.838297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:43.852917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eb4e7-c6d2-4ebe-b018-08f98327f057 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:43.920760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:43.921510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:43.921795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:43.922075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] PATH_INFO: `/v3/projects/d9e0dc09f9844b57bf7b38a5d4e54e17` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:43.923388 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e0dc09f9844b57bf7b38a5d4e54e17)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:43.925692 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:43.963352 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:43.978462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e0dc09f9844b57bf7b38a5d4e54e17', 'request_id': 'req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:43.985910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e0dc09f9844b57bf7b38a5d4e54e17', 'request_id': 'req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:43.991830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e0dc09f9844b57bf7b38a5d4e54e17', 'request_id': 'req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:44.004362 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] The token cache is being invalidate because project d9e0dc09f9844b57bf7b38a5d4e54e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:44.004648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:44.005372 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-131f9c85-04a0-4e8e-93eb-8aa85b8802e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:44.006934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1271/2591] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:43 2022] DELETE /identity/v3/projects/d9e0dc09f9844b57bf7b38a5d4e54e17 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:44.011912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375deb7f-5e4c-4397-809a-fde456911ff5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.012203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375deb7f-5e4c-4397-809a-fde456911ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.012482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375deb7f-5e4c-4397-809a-fde456911ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.034159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-375deb7f-5e4c-4397-809a-fde456911ff5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:44.035435 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-375deb7f-5e4c-4397-809a-fde456911ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:44.098695 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1321/2592] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:44.106606 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3d78-6880-41e6-90c9-6102c0a572fc None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.106896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3d78-6880-41e6-90c9-6102c0a572fc None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.107173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3d78-6880-41e6-90c9-6102c0a572fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.127180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-724f3d78-6880-41e6-90c9-6102c0a572fc None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:44.128385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-724f3d78-6880-41e6-90c9-6102c0a572fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:44.136625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1272/2593] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:44.141403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375deb7f-5e4c-4397-809a-fde456911ff5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.151551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375deb7f-5e4c-4397-809a-fde456911ff5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.157025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.157686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.157972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.158248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.158957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:44.159263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.160971 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.177751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1322/2594] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:45:44.178872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f3d78-6880-41e6-90c9-6102c0a572fc None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.188793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f3d78-6880-41e6-90c9-6102c0a572fc None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.193807 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c70ae-946a-42ea-83f7-9d2c83b075a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.194429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c70ae-946a-42ea-83f7-9d2c83b075a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.194714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c70ae-946a-42ea-83f7-9d2c83b075a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.195030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c70ae-946a-42ea-83f7-9d2c83b075a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.195700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c70ae-946a-42ea-83f7-9d2c83b075a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.197633 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c70ae-946a-42ea-83f7-9d2c83b075a8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.227370 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1273/2595] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:44 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:44.231305 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.241972 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceda91-f389-41e6-872d-7d00bc4fb522 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.318930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49424ef-8b6d-4eab-8897-009c1a02fe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.319557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49424ef-8b6d-4eab-8897-009c1a02fe15 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.319845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49424ef-8b6d-4eab-8897-009c1a02fe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.320450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49424ef-8b6d-4eab-8897-009c1a02fe15 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.320924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49424ef-8b6d-4eab-8897-009c1a02fe15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.322902 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49424ef-8b6d-4eab-8897-009c1a02fe15 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.332030 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:44.356259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1323/2596] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:44 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:44.360699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c70ae-946a-42ea-83f7-9d2c83b075a8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.371190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c70ae-946a-42ea-83f7-9d2c83b075a8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.380003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba06ab6-5848-4974-9de5-2162049fbc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.380737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06ab6-5848-4974-9de5-2162049fbc8b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.381096 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06ab6-5848-4974-9de5-2162049fbc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.381777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06ab6-5848-4974-9de5-2162049fbc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.382243 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba06ab6-5848-4974-9de5-2162049fbc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.383921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba06ab6-5848-4974-9de5-2162049fbc8b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.396139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1274/2597] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:44.400467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49424ef-8b6d-4eab-8897-009c1a02fe15 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.413161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49424ef-8b6d-4eab-8897-009c1a02fe15 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.417921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56070b-37f5-4c0e-82b9-cd428d92708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.418545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56070b-37f5-4c0e-82b9-cd428d92708d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.418831 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56070b-37f5-4c0e-82b9-cd428d92708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.419147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56070b-37f5-4c0e-82b9-cd428d92708d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.419811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56070b-37f5-4c0e-82b9-cd428d92708d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.422411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56070b-37f5-4c0e-82b9-cd428d92708d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.442151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1324/2598] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:45:44.446493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba06ab6-5848-4974-9de5-2162049fbc8b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.458533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba06ab6-5848-4974-9de5-2162049fbc8b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.463740 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f609ca-7ba8-42c4-b657-2ceda5334a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.464498 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f609ca-7ba8-42c4-b657-2ceda5334a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.464842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f609ca-7ba8-42c4-b657-2ceda5334a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.465190 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f609ca-7ba8-42c4-b657-2ceda5334a84 admin admin] PATH_INFO: `/v3/projects/ab77f957dd964914809643955e9f432f/users/41a4022d773a41f2898905f125d49e08/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.482092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f609ca-7ba8-42c4-b657-2ceda5334a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab77f957dd964914809643955e9f432f, user_id=41a4022d773a41f2898905f125d49e08, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.485725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f609ca-7ba8-42c4-b657-2ceda5334a84 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.502171 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1275/2599] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:44 2022] PUT /identity/v3/projects/ab77f957dd964914809643955e9f432f/users/41a4022d773a41f2898905f125d49e08/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:44.519573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f846c-f12f-45a9-8857-980a00e64850 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.519868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f846c-f12f-45a9-8857-980a00e64850 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.520124 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f846c-f12f-45a9-8857-980a00e64850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.562222 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-487f846c-f12f-45a9-8857-980a00e64850 None None] MFA Rules not processed for user `41a4022d773a41f2898905f125d49e08`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:44.563514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-487f846c-f12f-45a9-8857-980a00e64850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:44.627531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1325/2600] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:44.631957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f609ca-7ba8-42c4-b657-2ceda5334a84 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.697560 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f609ca-7ba8-42c4-b657-2ceda5334a84 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.703028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca076d-c948-4b9c-b1b2-e4b812ad188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.703741 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca076d-c948-4b9c-b1b2-e4b812ad188e admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.704090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca076d-c948-4b9c-b1b2-e4b812ad188e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.704368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca076d-c948-4b9c-b1b2-e4b812ad188e admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.705036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca076d-c948-4b9c-b1b2-e4b812ad188e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.706968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca076d-c948-4b9c-b1b2-e4b812ad188e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.738784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1276/2601] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:44 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:44.739995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f846c-f12f-45a9-8857-980a00e64850 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.751652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f846c-f12f-45a9-8857-980a00e64850 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.812908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1926d48-a9bb-433c-9ec3-3f5e63cce857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.813489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1926d48-a9bb-433c-9ec3-3f5e63cce857 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.813777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1926d48-a9bb-433c-9ec3-3f5e63cce857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.814053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1926d48-a9bb-433c-9ec3-3f5e63cce857 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.814713 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1926d48-a9bb-433c-9ec3-3f5e63cce857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.816866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1926d48-a9bb-433c-9ec3-3f5e63cce857 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.837866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1326/2602] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:44 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:44.842566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca076d-c948-4b9c-b1b2-e4b812ad188e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.868021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca076d-c948-4b9c-b1b2-e4b812ad188e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.874006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c594e1c-9ef8-480f-99e4-86d751edb0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.874634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594e1c-9ef8-480f-99e4-86d751edb0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.874920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594e1c-9ef8-480f-99e4-86d751edb0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.875199 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c594e1c-9ef8-480f-99e4-86d751edb0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.875899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c594e1c-9ef8-480f-99e4-86d751edb0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.877517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c594e1c-9ef8-480f-99e4-86d751edb0f3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.894749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1277/2603] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:44.899143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1926d48-a9bb-433c-9ec3-3f5e63cce857 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.910436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1926d48-a9bb-433c-9ec3-3f5e63cce857 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.917055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31222f0e-e828-4b1d-8236-f7692ce8c256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:44.917768 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31222f0e-e828-4b1d-8236-f7692ce8c256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:44.918052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31222f0e-e828-4b1d-8236-f7692ce8c256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:44.918327 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31222f0e-e828-4b1d-8236-f7692ce8c256 admin admin] PATH_INFO: `/v3/projects/ed30db84399b4a638e1ac6a2dcc25963/users/5936c23fe18941a48a10fa7575cc15af/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:44.936175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31222f0e-e828-4b1d-8236-f7692ce8c256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed30db84399b4a638e1ac6a2dcc25963, user_id=5936c23fe18941a48a10fa7575cc15af, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:44.938946 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31222f0e-e828-4b1d-8236-f7692ce8c256 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:44.952531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1327/2604] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:44 2022] PUT /identity/v3/projects/ed30db84399b4a638e1ac6a2dcc25963/users/5936c23fe18941a48a10fa7575cc15af/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:44.956592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c594e1c-9ef8-480f-99e4-86d751edb0f3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:44.966477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c594e1c-9ef8-480f-99e4-86d751edb0f3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:44.992877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:45.025719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ee2d2-41fc-4137-ae8e-4bb3b5749cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:45.026326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ee2d2-41fc-4137-ae8e-4bb3b5749cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:45.026609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ee2d2-41fc-4137-ae8e-4bb3b5749cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:45.026883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ee2d2-41fc-4137-ae8e-4bb3b5749cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:45.027568 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ee2d2-41fc-4137-ae8e-4bb3b5749cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:45.029246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ee2d2-41fc-4137-ae8e-4bb3b5749cf8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:45.038808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1278/2605] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:45.043181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31222f0e-e828-4b1d-8236-f7692ce8c256 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:45.053363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31222f0e-e828-4b1d-8236-f7692ce8c256 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:45.058158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:45.058813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:45.059093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:45.059364 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] PATH_INFO: `/v3/projects/ed30db84399b4a638e1ac6a2dcc25963/users/5936c23fe18941a48a10fa7575cc15af/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:45.062427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed30db84399b4a638e1ac6a2dcc25963, user_id=5936c23fe18941a48a10fa7575cc15af, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:45.065127 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:45.075466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1328/2606] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:45 2022] PUT /identity/v3/projects/ed30db84399b4a638e1ac6a2dcc25963/users/5936c23fe18941a48a10fa7575cc15af/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:45.087217 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b729-8f9e-408d-8e29-fbfbfdb3b095 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:45.087512 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b729-8f9e-408d-8e29-fbfbfdb3b095 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:45.087787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b729-8f9e-408d-8e29-fbfbfdb3b095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:45.115641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b714b729-8f9e-408d-8e29-fbfbfdb3b095 None None] MFA Rules not processed for user `5936c23fe18941a48a10fa7575cc15af`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:45.116873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b714b729-8f9e-408d-8e29-fbfbfdb3b095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:45.185019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1279/2607] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:45.198168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:45.199857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:45.230199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:45.263209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1da71-5f99-4598-854f-d65772d5ee5f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:45.267588 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8e6aa-2daa-4271-a90f-0e8afd8b16bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:45.268198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8e6aa-2daa-4271-a90f-0e8afd8b16bd service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:45.268481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8e6aa-2daa-4271-a90f-0e8afd8b16bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:45.268759 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8e6aa-2daa-4271-a90f-0e8afd8b16bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:45.278529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8e6aa-2daa-4271-a90f-0e8afd8b16bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:45.280413 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8e6aa-2daa-4271-a90f-0e8afd8b16bd service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:45.290882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1329/2608] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:45.320973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714b729-8f9e-408d-8e29-fbfbfdb3b095 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:45.387533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714b729-8f9e-408d-8e29-fbfbfdb3b095 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:45.392938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:45.393721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:45.394003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:45.394282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] PATH_INFO: `/v3/users/41a4022d773a41f2898905f125d49e08` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:45.395480 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a4022d773a41f2898905f125d49e08)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:45.397426 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:45.446938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a4022d773a41f2898905f125d49e08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:45.451801 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:45.456380 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a4022d773a41f2898905f125d49e08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:45.457395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a4022d773a41f2898905f125d49e08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:45.472215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a4022d773a41f2898905f125d49e08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:45.472852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a4022d773a41f2898905f125d49e08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:45.485410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1280/2609] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:45 2022] DELETE /identity/v3/users/41a4022d773a41f2898905f125d49e08 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:45.489985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3839b05-c01c-424b-9744-82efd8d4161b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:45.490281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3839b05-c01c-424b-9744-82efd8d4161b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:45.490562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3839b05-c01c-424b-9744-82efd8d4161b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:45.512803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d3839b05-c01c-424b-9744-82efd8d4161b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:45.514303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d3839b05-c01c-424b-9744-82efd8d4161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:45.583743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1330/2610] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:45.594140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:45.595391 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:45.653432 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5211ab5-b324-40e4-915e-b1ca1b03b244 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:45.657482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51f833-a47a-4b33-b131-8ea2f6747b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:45.658091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f833-a47a-4b33-b131-8ea2f6747b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:45.658369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f833-a47a-4b33-b131-8ea2f6747b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:45.658639 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f833-a47a-4b33-b131-8ea2f6747b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:45.667492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51f833-a47a-4b33-b131-8ea2f6747b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:45.668900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51f833-a47a-4b33-b131-8ea2f6747b78 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:45.678900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1281/2611] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:45.953264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3839b05-c01c-424b-9744-82efd8d4161b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:45.955021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d3839b05-c01c-424b-9744-82efd8d4161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:45.967670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3839b05-c01c-424b-9744-82efd8d4161b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:45.972700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:45.973111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:45.973434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:45.973709 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] PATH_INFO: `/v3/projects/ab77f957dd964914809643955e9f432f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:45.974885 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab77f957dd964914809643955e9f432f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:45.976768 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:46.025334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab77f957dd964914809643955e9f432f', 'request_id': 'req-3a1133ab-7e6c-4a40-bb56-8aea090d22af'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.031176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab77f957dd964914809643955e9f432f', 'request_id': 'req-3a1133ab-7e6c-4a40-bb56-8aea090d22af'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.037824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab77f957dd964914809643955e9f432f', 'request_id': 'req-3a1133ab-7e6c-4a40-bb56-8aea090d22af'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.049500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] The token cache is being invalidate because project ab77f957dd964914809643955e9f432f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:46.049789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.050416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.051921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1331/2612] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:45 2022] DELETE /identity/v3/projects/ab77f957dd964914809643955e9f432f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:46.055928 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51f833-a47a-4b33-b131-8ea2f6747b78 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:46.066726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51f833-a47a-4b33-b131-8ea2f6747b78 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:46.121556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:46.122120 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:46.122405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:46.122683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] PATH_INFO: `/v3/users/5936c23fe18941a48a10fa7575cc15af` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:46.124430 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5936c23fe18941a48a10fa7575cc15af)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:46.126377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:46.171472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5936c23fe18941a48a10fa7575cc15af'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.178471 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5936c23fe18941a48a10fa7575cc15af'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.179069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5936c23fe18941a48a10fa7575cc15af'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.191690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5936c23fe18941a48a10fa7575cc15af'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.192309 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db49c30c-0f77-428d-a127-b783f3b28b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5936c23fe18941a48a10fa7575cc15af'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.194269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:46.207027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1282/2613] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:46 2022] DELETE /identity/v3/users/5936c23fe18941a48a10fa7575cc15af => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:46.480884 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:46.482720 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:46.497569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1133ab-7e6c-4a40-bb56-8aea090d22af admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:46.554531 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:46.569798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:46.570503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:46.570784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:46.571056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] PATH_INFO: `/v3/projects/ed30db84399b4a638e1ac6a2dcc25963` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:46.572369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed30db84399b4a638e1ac6a2dcc25963)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:46.574451 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:46.634552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed30db84399b4a638e1ac6a2dcc25963', 'request_id': 'req-40e3d3c4-3313-4eda-8737-7429590b746c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.643660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed30db84399b4a638e1ac6a2dcc25963', 'request_id': 'req-40e3d3c4-3313-4eda-8737-7429590b746c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.652594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed30db84399b4a638e1ac6a2dcc25963', 'request_id': 'req-40e3d3c4-3313-4eda-8737-7429590b746c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.664895 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] The token cache is being invalidate because project ed30db84399b4a638e1ac6a2dcc25963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:46.665186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.665917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-40e3d3c4-3313-4eda-8737-7429590b746c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:46.667418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1332/2614] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:46 2022] DELETE /identity/v3/projects/ed30db84399b4a638e1ac6a2dcc25963 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:46.673184 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9022f7-afd9-42f5-81c2-367a4f634b99 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:46.673677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9022f7-afd9-42f5-81c2-367a4f634b99 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:46.674099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9022f7-afd9-42f5-81c2-367a4f634b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:46.694552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4f9022f7-afd9-42f5-81c2-367a4f634b99 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:46.695775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4f9022f7-afd9-42f5-81c2-367a4f634b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:46.761262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1283/2615] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:46.770038 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88f8a8-ed2d-4732-a01c-49c931f4fbc5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:46.770393 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88f8a8-ed2d-4732-a01c-49c931f4fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:46.770733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88f8a8-ed2d-4732-a01c-49c931f4fbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:46.790854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8b88f8a8-ed2d-4732-a01c-49c931f4fbc5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:46.792043 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8b88f8a8-ed2d-4732-a01c-49c931f4fbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:46.800668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1333/2616] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:46.804923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9022f7-afd9-42f5-81c2-367a4f634b99 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:46.815065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9022f7-afd9-42f5-81c2-367a4f634b99 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:46.819936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:46.820557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:46.820841 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:46.821118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:46.821860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:46.822172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:46.823754 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:46.835839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1284/2617] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:45:46.840082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88f8a8-ed2d-4732-a01c-49c931f4fbc5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:46.849874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88f8a8-ed2d-4732-a01c-49c931f4fbc5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:46.854525 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd26b-bf3c-4a5e-af14-50dac4525c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:46.855123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd26b-bf3c-4a5e-af14-50dac4525c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:46.855399 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd26b-bf3c-4a5e-af14-50dac4525c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:46.855666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd26b-bf3c-4a5e-af14-50dac4525c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:46.856328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddd26b-bf3c-4a5e-af14-50dac4525c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:46.858403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddd26b-bf3c-4a5e-af14-50dac4525c29 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:46.884941 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1334/2618] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:46 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:45:46.889762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:46.899698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a291f-9399-4f50-a267-76b795d88479 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:46.955883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f944-062e-4c88-8ff7-ac9c9b422f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:46.956422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f944-062e-4c88-8ff7-ac9c9b422f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:46.956707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f944-062e-4c88-8ff7-ac9c9b422f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:46.956985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f944-062e-4c88-8ff7-ac9c9b422f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:46.957705 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917f944-062e-4c88-8ff7-ac9c9b422f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:46.959617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917f944-062e-4c88-8ff7-ac9c9b422f48 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:46.981104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1285/2619] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:46 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:46.985465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd26b-bf3c-4a5e-af14-50dac4525c29 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:46.995915 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd26b-bf3c-4a5e-af14-50dac4525c29 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.000606 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018b555-369b-4441-a168-cae623a5efc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.001231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b555-369b-4441-a168-cae623a5efc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.001524 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b555-369b-4441-a168-cae623a5efc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.001800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b555-369b-4441-a168-cae623a5efc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.002489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018b555-369b-4441-a168-cae623a5efc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.004332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018b555-369b-4441-a168-cae623a5efc1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.014819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1335/2620] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:47.018792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f944-062e-4c88-8ff7-ac9c9b422f48 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.030079 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f944-062e-4c88-8ff7-ac9c9b422f48 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.035027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa4e85-7dab-4dc6-b8f5-10d538da511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.036231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4e85-7dab-4dc6-b8f5-10d538da511b admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.036500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4e85-7dab-4dc6-b8f5-10d538da511b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.036806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4e85-7dab-4dc6-b8f5-10d538da511b admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.037700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa4e85-7dab-4dc6-b8f5-10d538da511b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.039299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa4e85-7dab-4dc6-b8f5-10d538da511b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.050514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1286/2621] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:47.054914 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018b555-369b-4441-a168-cae623a5efc1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.066351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018b555-369b-4441-a168-cae623a5efc1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.071042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a020c2a-4d91-4380-9ff8-1a5622b7e338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.071868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a020c2a-4d91-4380-9ff8-1a5622b7e338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.072149 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a020c2a-4d91-4380-9ff8-1a5622b7e338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.072423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a020c2a-4d91-4380-9ff8-1a5622b7e338 admin admin] PATH_INFO: `/v3/projects/d803ee5c9c414a48b03cbb3dfb29cceb/users/fcfeecda6dee49b6ad0a7fcc32a02feb/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.088673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a020c2a-4d91-4380-9ff8-1a5622b7e338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d803ee5c9c414a48b03cbb3dfb29cceb, user_id=fcfeecda6dee49b6ad0a7fcc32a02feb, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.091369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a020c2a-4d91-4380-9ff8-1a5622b7e338 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.106477 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1336/2622] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:47 2022] PUT /identity/v3/projects/d803ee5c9c414a48b03cbb3dfb29cceb/users/fcfeecda6dee49b6ad0a7fcc32a02feb/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:45:47.118447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1233693-0c79-4c49-924c-d3bffef30b8c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.118975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1233693-0c79-4c49-924c-d3bffef30b8c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.118975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1233693-0c79-4c49-924c-d3bffef30b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.148310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-b1233693-0c79-4c49-924c-d3bffef30b8c None None] MFA Rules not processed for user `fcfeecda6dee49b6ad0a7fcc32a02feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:47.149814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-b1233693-0c79-4c49-924c-d3bffef30b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:47.196511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:47.218024 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1287/2623] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:47.222501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a020c2a-4d91-4380-9ff8-1a5622b7e338 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.283023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a020c2a-4d91-4380-9ff8-1a5622b7e338 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.287824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394a10a-cb79-44f2-9eea-1965a0350df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.288437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394a10a-cb79-44f2-9eea-1965a0350df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.288723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394a10a-cb79-44f2-9eea-1965a0350df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.288999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394a10a-cb79-44f2-9eea-1965a0350df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.289692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394a10a-cb79-44f2-9eea-1965a0350df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.291585 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394a10a-cb79-44f2-9eea-1965a0350df2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.312674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:47.323155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1337/2624] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:47 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:47.324304 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1233693-0c79-4c49-924c-d3bffef30b8c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.335467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1233693-0c79-4c49-924c-d3bffef30b8c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.391315 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5a18-af4e-4fad-a4db-2c1dfa3598b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.391921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5a18-af4e-4fad-a4db-2c1dfa3598b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.392206 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5a18-af4e-4fad-a4db-2c1dfa3598b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.392477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5a18-af4e-4fad-a4db-2c1dfa3598b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.393129 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac5a18-af4e-4fad-a4db-2c1dfa3598b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.395049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac5a18-af4e-4fad-a4db-2c1dfa3598b7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.417151 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1288/2625] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:47 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:47.422117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394a10a-cb79-44f2-9eea-1965a0350df2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.432666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394a10a-cb79-44f2-9eea-1965a0350df2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.437940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddc560-1af9-4ba5-9a29-78e394b297ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.438569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddc560-1af9-4ba5-9a29-78e394b297ae admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.438854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddc560-1af9-4ba5-9a29-78e394b297ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.439130 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddc560-1af9-4ba5-9a29-78e394b297ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.439830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddc560-1af9-4ba5-9a29-78e394b297ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.441806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddc560-1af9-4ba5-9a29-78e394b297ae admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.452812 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1338/2626] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:47.456813 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5a18-af4e-4fad-a4db-2c1dfa3598b7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.467384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5a18-af4e-4fad-a4db-2c1dfa3598b7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.472019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089fd1e-697f-4742-b568-fe6102cb3469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.472618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089fd1e-697f-4742-b568-fe6102cb3469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.472902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089fd1e-697f-4742-b568-fe6102cb3469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.473181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089fd1e-697f-4742-b568-fe6102cb3469 admin admin] PATH_INFO: `/v3/projects/82df92c6e1b241dbab23df701cdc1e7f/users/ac896fd4ed294953ab672ded3c11bb11/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.488303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089fd1e-697f-4742-b568-fe6102cb3469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82df92c6e1b241dbab23df701cdc1e7f, user_id=ac896fd4ed294953ab672ded3c11bb11, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.490978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089fd1e-697f-4742-b568-fe6102cb3469 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.503590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1289/2627] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:47 2022] PUT /identity/v3/projects/82df92c6e1b241dbab23df701cdc1e7f/users/ac896fd4ed294953ab672ded3c11bb11/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:47.507751 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddc560-1af9-4ba5-9a29-78e394b297ae admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.518612 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddc560-1af9-4ba5-9a29-78e394b297ae admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.575403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980bab3-5354-431b-95f4-92df4a335bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.576747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bab3-5354-431b-95f4-92df4a335bac admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.577050 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bab3-5354-431b-95f4-92df4a335bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.577368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bab3-5354-431b-95f4-92df4a335bac admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.578082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980bab3-5354-431b-95f4-92df4a335bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.579789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980bab3-5354-431b-95f4-92df4a335bac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.589521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1339/2628] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:47.594677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089fd1e-697f-4742-b568-fe6102cb3469 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.605382 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089fd1e-697f-4742-b568-fe6102cb3469 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.610069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.610721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.611002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.611301 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] PATH_INFO: `/v3/projects/82df92c6e1b241dbab23df701cdc1e7f/users/ac896fd4ed294953ab672ded3c11bb11/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.613197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82df92c6e1b241dbab23df701cdc1e7f, user_id=ac896fd4ed294953ab672ded3c11bb11, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.617318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.629084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1290/2629] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:47 2022] PUT /identity/v3/projects/82df92c6e1b241dbab23df701cdc1e7f/users/ac896fd4ed294953ab672ded3c11bb11/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:47.641847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6d83-5bd4-4fac-a25d-ce9af2c68179 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.642303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6d83-5bd4-4fac-a25d-ce9af2c68179 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.642466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6d83-5bd4-4fac-a25d-ce9af2c68179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.672033 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1ecd6d83-5bd4-4fac-a25d-ce9af2c68179 None None] MFA Rules not processed for user `ac896fd4ed294953ab672ded3c11bb11`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:47.673257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1ecd6d83-5bd4-4fac-a25d-ce9af2c68179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:47.746993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1340/2630] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:47.759898 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.761534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:47.858398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd369236-8c52-41cb-83b3-5cfc51f8daf4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.865463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf214f-aa31-4fe4-bc95-68bfb8d29666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.866182 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf214f-aa31-4fe4-bc95-68bfb8d29666 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.866574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf214f-aa31-4fe4-bc95-68bfb8d29666 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.866960 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf214f-aa31-4fe4-bc95-68bfb8d29666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:47.878922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf214f-aa31-4fe4-bc95-68bfb8d29666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:47.880334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf214f-aa31-4fe4-bc95-68bfb8d29666 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:47.893338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1291/2631] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:47.942159 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd6d83-5bd4-4fac-a25d-ce9af2c68179 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:47.943565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1ecd6d83-5bd4-4fac-a25d-ce9af2c68179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:47.956248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd6d83-5bd4-4fac-a25d-ce9af2c68179 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:47.959857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf54cb-75b5-4a13-b8f5-aaf08b6fd5a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:47.960474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf54cb-75b5-4a13-b8f5-aaf08b6fd5a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:47.960750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf54cb-75b5-4a13-b8f5-aaf08b6fd5a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:47.961025 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf54cb-75b5-4a13-b8f5-aaf08b6fd5a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:48.019809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf54cb-75b5-4a13-b8f5-aaf08b6fd5a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:48.021147 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf54cb-75b5-4a13-b8f5-aaf08b6fd5a7 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:48.030501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1341/2632] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:51.432671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf214f-aa31-4fe4-bc95-68bfb8d29666 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:51.479929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:51.499891 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf214f-aa31-4fe4-bc95-68bfb8d29666 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:51.504407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:51.505045 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:51.505434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:51.505766 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] PATH_INFO: `/v3/users/fcfeecda6dee49b6ad0a7fcc32a02feb` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:51.506875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcfeecda6dee49b6ad0a7fcc32a02feb)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:51.508784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:51.557478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcfeecda6dee49b6ad0a7fcc32a02feb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:51.567485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcfeecda6dee49b6ad0a7fcc32a02feb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:51.568114 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcfeecda6dee49b6ad0a7fcc32a02feb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:51.582006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcfeecda6dee49b6ad0a7fcc32a02feb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:51.582681 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcfeecda6dee49b6ad0a7fcc32a02feb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:51.595464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1292/2633] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:51 2022] DELETE /identity/v3/users/fcfeecda6dee49b6ad0a7fcc32a02feb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:51.600773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79849fe9-f3e9-4914-a041-cc709f76a19b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:51.601161 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79849fe9-f3e9-4914-a041-cc709f76a19b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:51.601257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79849fe9-f3e9-4914-a041-cc709f76a19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:51.621501 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-79849fe9-f3e9-4914-a041-cc709f76a19b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:51.623031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-79849fe9-f3e9-4914-a041-cc709f76a19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:51.628505 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:51.690545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1342/2634] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:45:51.700517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:51.702648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:51.766064 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408ca8f-0dc2-4c19-a762-506098a20883 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:51.769964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b10f1-e67e-40da-a92e-c475005943bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:51.770585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b10f1-e67e-40da-a92e-c475005943bb service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:51.770870 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b10f1-e67e-40da-a92e-c475005943bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:51.771147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b10f1-e67e-40da-a92e-c475005943bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:51.781239 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b10f1-e67e-40da-a92e-c475005943bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:51.782987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b10f1-e67e-40da-a92e-c475005943bb service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:51.793003 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1293/2635] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:52.092793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79849fe9-f3e9-4914-a041-cc709f76a19b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:52.094229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-79849fe9-f3e9-4914-a041-cc709f76a19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:52.106232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79849fe9-f3e9-4914-a041-cc709f76a19b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:52.122721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:52.124432 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:52.124834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:52.125191 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] PATH_INFO: `/v3/projects/d803ee5c9c414a48b03cbb3dfb29cceb` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:52.126842 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d803ee5c9c414a48b03cbb3dfb29cceb)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:52.128743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:52.179115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd803ee5c9c414a48b03cbb3dfb29cceb', 'request_id': 'req-f85655f0-8534-45d6-b750-15226633d924'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.184821 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd803ee5c9c414a48b03cbb3dfb29cceb', 'request_id': 'req-f85655f0-8534-45d6-b750-15226633d924'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.199540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd803ee5c9c414a48b03cbb3dfb29cceb', 'request_id': 'req-f85655f0-8534-45d6-b750-15226633d924'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.210247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b10f1-e67e-40da-a92e-c475005943bb service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:52.211483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] The token cache is being invalidate because project d803ee5c9c414a48b03cbb3dfb29cceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:52.211791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.212441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.214013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1343/2636] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:52 2022] DELETE /identity/v3/projects/d803ee5c9c414a48b03cbb3dfb29cceb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:52.217969 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:52.219324 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:52.219727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b10f1-e67e-40da-a92e-c475005943bb service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:52.273779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:52.274441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:52.274721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:52.274995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:52.276739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:52.284285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85655f0-8534-45d6-b750-15226633d924 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:52.289094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:52.290026 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:52.290312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:52.290586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] PATH_INFO: `/v3/users/ac896fd4ed294953ab672ded3c11bb11` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:52.291961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac896fd4ed294953ab672ded3c11bb11)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:52.293967 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:52.298541 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:52.334269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:52.345839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac896fd4ed294953ab672ded3c11bb11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.351406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:52.352666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:52.353110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac896fd4ed294953ab672ded3c11bb11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.361276 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac896fd4ed294953ab672ded3c11bb11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.361986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac896fd4ed294953ab672ded3c11bb11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.363175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1294/2637] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:45:52.374924 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-91210f08-d729-4cf6-b42c-53c0e0f8fef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac896fd4ed294953ab672ded3c11bb11'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.381013 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1344/2638] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:52 2022] DELETE /identity/v3/users/ac896fd4ed294953ab672ded3c11bb11 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:45:52.810753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:52.822253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3939560-54e6-4481-ad5b-25e36ff3d495 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:52.885618 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:52.886271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:52.886555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:52.886830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] PATH_INFO: `/v3/projects/82df92c6e1b241dbab23df701cdc1e7f` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:52.888005 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82df92c6e1b241dbab23df701cdc1e7f)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:52.890491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:52.941962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82df92c6e1b241dbab23df701cdc1e7f', 'request_id': 'req-522783a8-292a-4ac3-ba9c-97035b899e32'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.950500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82df92c6e1b241dbab23df701cdc1e7f', 'request_id': 'req-522783a8-292a-4ac3-ba9c-97035b899e32'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.956919 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82df92c6e1b241dbab23df701cdc1e7f', 'request_id': 'req-522783a8-292a-4ac3-ba9c-97035b899e32'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.970929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] The token cache is being invalidate because project 82df92c6e1b241dbab23df701cdc1e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:52.971274 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.971991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-522783a8-292a-4ac3-ba9c-97035b899e32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:52.973992 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1295/2639] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:52 2022] DELETE /identity/v3/projects/82df92c6e1b241dbab23df701cdc1e7f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:52.979252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71debaf-fae4-4757-b467-074f2b6c3be2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:52.979726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71debaf-fae4-4757-b467-074f2b6c3be2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:52.979845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71debaf-fae4-4757-b467-074f2b6c3be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.001017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b71debaf-fae4-4757-b467-074f2b6c3be2 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:53.002275 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b71debaf-fae4-4757-b467-074f2b6c3be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:53.070301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1345/2640] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:53.080630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f1b74-b1e5-49a9-8b71-08979e483242 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.081088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f1b74-b1e5-49a9-8b71-08979e483242 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.081478 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f1b74-b1e5-49a9-8b71-08979e483242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.104802 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-490f1b74-b1e5-49a9-8b71-08979e483242 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:53.106335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-490f1b74-b1e5-49a9-8b71-08979e483242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:53.114996 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1296/2641] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:53.119229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71debaf-fae4-4757-b467-074f2b6c3be2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.129358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71debaf-fae4-4757-b467-074f2b6c3be2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.134158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.134858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.135221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.135590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.136397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:53.136777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.139021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.156272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1346/2642] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:45:53.157428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f1b74-b1e5-49a9-8b71-08979e483242 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.167316 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f1b74-b1e5-49a9-8b71-08979e483242 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.172246 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e34a4e-3cfe-4a63-a51e-84bddd79ddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.172857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e34a4e-3cfe-4a63-a51e-84bddd79ddaa admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.173140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e34a4e-3cfe-4a63-a51e-84bddd79ddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.173452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e34a4e-3cfe-4a63-a51e-84bddd79ddaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.174192 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e34a4e-3cfe-4a63-a51e-84bddd79ddaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.176110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e34a4e-3cfe-4a63-a51e-84bddd79ddaa admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.209726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1297/2643] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:53 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:45:53.210959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.221530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f623e-ed8d-4273-b84f-a3d851e91c3f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.281375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f4703-50bf-4446-b12a-15dc086c3c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.282011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4703-50bf-4446-b12a-15dc086c3c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.282297 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4703-50bf-4446-b12a-15dc086c3c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.282576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4703-50bf-4446-b12a-15dc086c3c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.283248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f4703-50bf-4446-b12a-15dc086c3c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.285207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f4703-50bf-4446-b12a-15dc086c3c39 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.308593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1347/2644] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:53 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:45:53.312515 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e34a4e-3cfe-4a63-a51e-84bddd79ddaa admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.323103 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e34a4e-3cfe-4a63-a51e-84bddd79ddaa admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.328927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b7e73-cce5-461c-8634-5d4e4da72839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.329587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b7e73-cce5-461c-8634-5d4e4da72839 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.329868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b7e73-cce5-461c-8634-5d4e4da72839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.330146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b7e73-cce5-461c-8634-5d4e4da72839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.330842 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b7e73-cce5-461c-8634-5d4e4da72839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.332439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b7e73-cce5-461c-8634-5d4e4da72839 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.343227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1298/2645] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:53.347412 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f4703-50bf-4446-b12a-15dc086c3c39 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.358210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f4703-50bf-4446-b12a-15dc086c3c39 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.363079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15149795-f650-4652-9515-20ff923c3e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.363704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15149795-f650-4652-9515-20ff923c3e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.363983 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15149795-f650-4652-9515-20ff923c3e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.364255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15149795-f650-4652-9515-20ff923c3e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.364951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15149795-f650-4652-9515-20ff923c3e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.367179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15149795-f650-4652-9515-20ff923c3e77 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.379546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1348/2646] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:53.384061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b7e73-cce5-461c-8634-5d4e4da72839 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.390720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:53.396501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b7e73-cce5-461c-8634-5d4e4da72839 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.402957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9127e-35c4-497a-8730-46a3c2fd9d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.403721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9127e-35c4-497a-8730-46a3c2fd9d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.404901 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9127e-35c4-497a-8730-46a3c2fd9d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.405339 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9127e-35c4-497a-8730-46a3c2fd9d25 admin admin] PATH_INFO: `/v3/projects/118b72df89d7441cb581f10755acc574/users/9491acfc0a3547259f3d540c1aa78595/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.421166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9127e-35c4-497a-8730-46a3c2fd9d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118b72df89d7441cb581f10755acc574, user_id=9491acfc0a3547259f3d540c1aa78595, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.423831 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9127e-35c4-497a-8730-46a3c2fd9d25 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.442604 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1299/2647] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:53 2022] PUT /identity/v3/projects/118b72df89d7441cb581f10755acc574/users/9491acfc0a3547259f3d540c1aa78595/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:53.455678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c46f-cd8f-4aaa-8e3d-2ac374fb74fd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.455978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c46f-cd8f-4aaa-8e3d-2ac374fb74fd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.456633 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c46f-cd8f-4aaa-8e3d-2ac374fb74fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.487560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9da3c46f-cd8f-4aaa-8e3d-2ac374fb74fd None None] MFA Rules not processed for user `9491acfc0a3547259f3d540c1aa78595`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:53.488934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9da3c46f-cd8f-4aaa-8e3d-2ac374fb74fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:53.494600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:53.560719 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1349/2648] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:53.564979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9127e-35c4-497a-8730-46a3c2fd9d25 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.631463 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9127e-35c4-497a-8730-46a3c2fd9d25 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.636188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058832cb-8184-49ea-8a49-899e9b51380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.637070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058832cb-8184-49ea-8a49-899e9b51380b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.637392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058832cb-8184-49ea-8a49-899e9b51380b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.637676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058832cb-8184-49ea-8a49-899e9b51380b admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.639049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058832cb-8184-49ea-8a49-899e9b51380b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.641058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058832cb-8184-49ea-8a49-899e9b51380b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.671437 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1300/2649] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:53 2022] POST /identity/v3/projects => generated 407 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:45:53.675551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3c46f-cd8f-4aaa-8e3d-2ac374fb74fd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.686417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3c46f-cd8f-4aaa-8e3d-2ac374fb74fd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.745143 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.745809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.746094 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.746371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.747047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.749029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.772886 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1350/2650] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:53 2022] POST /identity/v3/users => generated 310 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:45:53.777241 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058832cb-8184-49ea-8a49-899e9b51380b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.789723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058832cb-8184-49ea-8a49-899e9b51380b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.795189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c0a04-f665-4716-a2b0-a5d324a1dc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.795835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c0a04-f665-4716-a2b0-a5d324a1dc65 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.796119 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c0a04-f665-4716-a2b0-a5d324a1dc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.796394 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c0a04-f665-4716-a2b0-a5d324a1dc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.797104 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c0a04-f665-4716-a2b0-a5d324a1dc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.798760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c0a04-f665-4716-a2b0-a5d324a1dc65 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.808731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.810734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:53.812933 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1301/2651] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:45:53.817135 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c0a04-f665-4716-a2b0-a5d324a1dc65 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.824122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e392aa-029e-4014-9f7c-dec4421c9001 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.829474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c0a04-f665-4716-a2b0-a5d324a1dc65 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.829648 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.830272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.830560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.830845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] PATH_INFO: `/v3/users/efd0eb7116e04b76ac5f522fca889f87` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.832098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd0eb7116e04b76ac5f522fca889f87)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.834722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.835779 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e638f20-f9e0-4026-bf7c-1fce2c099f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:53.836678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e638f20-f9e0-4026-bf7c-1fce2c099f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.837248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e638f20-f9e0-4026-bf7c-1fce2c099f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.837763 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e638f20-f9e0-4026-bf7c-1fce2c099f39 admin admin] PATH_INFO: `/v3/projects/eff99f1ef72d422795776c3e324f7b7e/users/7c176182ac6f41f586b69ea58218dcd3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.855285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e638f20-f9e0-4026-bf7c-1fce2c099f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff99f1ef72d422795776c3e324f7b7e, user_id=7c176182ac6f41f586b69ea58218dcd3, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:53.858367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e638f20-f9e0-4026-bf7c-1fce2c099f39 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:53.878091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1302/2652] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:53 2022] PUT /identity/v3/projects/eff99f1ef72d422795776c3e324f7b7e/users/7c176182ac6f41f586b69ea58218dcd3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:53.881987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e638f20-f9e0-4026-bf7c-1fce2c099f39 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:53.892906 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd0eb7116e04b76ac5f522fca889f87'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:53.901675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd0eb7116e04b76ac5f522fca889f87'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:53.914392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd0eb7116e04b76ac5f522fca889f87'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:53.916036 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd0eb7116e04b76ac5f522fca889f87'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:53.944166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbcd84af-57dd-4821-8140-a89e8906de30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd0eb7116e04b76ac5f522fca889f87'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:53.953340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1351/2653] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:45:53 2022] DELETE /identity/v3/users/efd0eb7116e04b76ac5f522fca889f87 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:53.956626 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001a03-edb4-479e-a7ad-ade7cb30029a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:53.958097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001a03-edb4-479e-a7ad-ade7cb30029a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:53.958416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001a03-edb4-479e-a7ad-ade7cb30029a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:53.970248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e638f20-f9e0-4026-bf7c-1fce2c099f39 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:53.993044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-04001a03-edb4-479e-a7ad-ade7cb30029a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:53.995202 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-04001a03-edb4-479e-a7ad-ade7cb30029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.001522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:54.010479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:54.042893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:54.043635 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.043983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.044266 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.045616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:54.047326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:54.061573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1303/2654] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:45:54.062786 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:54.065075 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.068261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1352/2655] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:45:54.078044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc02b9-83a4-4526-a71f-9c1e2eb26ae3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:54.078438 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04001a03-edb4-479e-a7ad-ade7cb30029a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:54.080439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-04001a03-edb4-479e-a7ad-ade7cb30029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.082669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdab95-697d-49cc-a657-d9bf03ebeab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:54.083340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdab95-697d-49cc-a657-d9bf03ebeab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.083654 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdab95-697d-49cc-a657-d9bf03ebeab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.083900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdab95-697d-49cc-a657-d9bf03ebeab5 admin admin] PATH_INFO: `/v3/projects/eff99f1ef72d422795776c3e324f7b7e/users/7c176182ac6f41f586b69ea58218dcd3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.085969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdab95-697d-49cc-a657-d9bf03ebeab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff99f1ef72d422795776c3e324f7b7e, user_id=7c176182ac6f41f586b69ea58218dcd3, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:54.088599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdab95-697d-49cc-a657-d9bf03ebeab5 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:54.100530 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1304/2656] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:54 2022] PUT /identity/v3/projects/eff99f1ef72d422795776c3e324f7b7e/users/7c176182ac6f41f586b69ea58218dcd3/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:54.111665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965816-510e-4076-a2c2-434018ae48ac None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.111665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965816-510e-4076-a2c2-434018ae48ac None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.111968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965816-510e-4076-a2c2-434018ae48ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.144376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-d8965816-510e-4076-a2c2-434018ae48ac None None] MFA Rules not processed for user `7c176182ac6f41f586b69ea58218dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:54.145610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d8965816-510e-4076-a2c2-434018ae48ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.148278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04001a03-edb4-479e-a7ad-ade7cb30029a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:54.204620 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9cc51-d4e2-420f-a6db-6b9873eebab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:54.205466 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9cc51-d4e2-420f-a6db-6b9873eebab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.205879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9cc51-d4e2-420f-a6db-6b9873eebab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.206251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9cc51-d4e2-420f-a6db-6b9873eebab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.226507 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1305/2657] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:54 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:45:54.237269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8965816-510e-4076-a2c2-434018ae48ac None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:54.238682 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d8965816-510e-4076-a2c2-434018ae48ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.276380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9cc51-d4e2-420f-a6db-6b9873eebab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:54.278027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9cc51-d4e2-420f-a6db-6b9873eebab5 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:54.288255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1353/2658] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:45:54.322409 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8965816-510e-4076-a2c2-434018ae48ac None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:54.327824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:54.328760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.329159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.329570 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.343673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:54.345044 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:54.356023 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1306/2659] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:54 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:45:54.612292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9cc51-d4e2-420f-a6db-6b9873eebab5 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:54.615733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:54.617237 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.629212 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9cc51-d4e2-420f-a6db-6b9873eebab5 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:54.635300 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d004da1-d6ab-483f-a764-89e491e9da21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:54.638328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004da1-d6ab-483f-a764-89e491e9da21 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.640051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68161e-eede-46dd-b676-4fd37216193b service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:54.646056 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:54.646363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004da1-d6ab-483f-a764-89e491e9da21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.646731 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004da1-d6ab-483f-a764-89e491e9da21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.647389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.647720 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.648000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] PATH_INFO: `/v3/projects/21571203026444339b747739c5bed02a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.649599 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] RBAC: Authorizing `identity:delete_project(project_id=21571203026444339b747739c5bed02a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:54.650164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1d004da1-d6ab-483f-a764-89e491e9da21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.651523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:54.666031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:54.720194 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:54.728700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21571203026444339b747739c5bed02a', 'request_id': 'req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:54.741164 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d004da1-d6ab-483f-a764-89e491e9da21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:54.743560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d004da1-d6ab-483f-a764-89e491e9da21 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:54.749906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21571203026444339b747739c5bed02a', 'request_id': 'req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:54.759621 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21571203026444339b747739c5bed02a', 'request_id': 'req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:54.762766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1354/2660] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:54 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:54.789982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] The token cache is being invalidate because project 21571203026444339b747739c5bed02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:45:54.790323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:54.791051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c4564e61-26bb-4f86-b3e0-77e2f88ad48e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:45:54.792979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1307/2661] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:45:54 2022] DELETE /identity/v3/projects/21571203026444339b747739c5bed02a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:54.798880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde4f97-daa9-4146-a05f-0dea60a6baa1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.799203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde4f97-daa9-4146-a05f-0dea60a6baa1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.799486 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde4f97-daa9-4146-a05f-0dea60a6baa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.823817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3bde4f97-daa9-4146-a05f-0dea60a6baa1 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:54.825203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3bde4f97-daa9-4146-a05f-0dea60a6baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.907436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1355/2662] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:54.919400 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aa3ae-073a-4463-9af5-975ab87194ae None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.920188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aa3ae-073a-4463-9af5-975ab87194ae None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.920499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aa3ae-073a-4463-9af5-975ab87194ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.948026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-e25aa3ae-073a-4463-9af5-975ab87194ae None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:54.949452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e25aa3ae-073a-4463-9af5-975ab87194ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:54.967034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1308/2663] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:54.967743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde4f97-daa9-4146-a05f-0dea60a6baa1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:54.980325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde4f97-daa9-4146-a05f-0dea60a6baa1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:54.994108 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:54.994894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:54.995737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:54.996112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:54.998089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:45:54.998891 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:55.001093 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:55.021992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1356/2664] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:45:55.023899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aa3ae-073a-4463-9af5-975ab87194ae None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:55.037270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aa3ae-073a-4463-9af5-975ab87194ae None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:55.044186 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd2485-2f51-405a-8d97-be4153b446c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:55.045603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd2485-2f51-405a-8d97-be4153b446c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:55.046029 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd2485-2f51-405a-8d97-be4153b446c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:55.046474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd2485-2f51-405a-8d97-be4153b446c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:55.047292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd2485-2f51-405a-8d97-be4153b446c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:55.049563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd2485-2f51-405a-8d97-be4153b446c4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:55.088958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1309/2665] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:55 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:55.094341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:55.107723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98db0b3-caa1-4a80-8a9f-921add5585b0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:55.187247 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d6377-9845-497b-815b-ce499333c93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:55.187992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d6377-9845-497b-815b-ce499333c93a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:55.188349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d6377-9845-497b-815b-ce499333c93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:55.189455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d6377-9845-497b-815b-ce499333c93a admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:55.189668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d6377-9845-497b-815b-ce499333c93a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:55.192236 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d6377-9845-497b-815b-ce499333c93a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:55.226375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1357/2666] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:55 2022] POST /identity/v3/users => generated 286 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:55.230403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd2485-2f51-405a-8d97-be4153b446c4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:55.242431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd2485-2f51-405a-8d97-be4153b446c4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:55.249866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9706f7-854a-446c-9922-ccca79153a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:55.250537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9706f7-854a-446c-9922-ccca79153a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:55.250821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9706f7-854a-446c-9922-ccca79153a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:55.251093 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9706f7-854a-446c-9922-ccca79153a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:55.251838 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9706f7-854a-446c-9922-ccca79153a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:55.254008 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9706f7-854a-446c-9922-ccca79153a3d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:55.268593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1310/2667] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:55.272874 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d6377-9845-497b-815b-ce499333c93a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:55.284734 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d6377-9845-497b-815b-ce499333c93a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:55.290441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa8c01-a165-4fa2-a388-26daed74fb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:55.291540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa8c01-a165-4fa2-a388-26daed74fb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:55.291859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa8c01-a165-4fa2-a388-26daed74fb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:55.292137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa8c01-a165-4fa2-a388-26daed74fb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:55.293103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa8c01-a165-4fa2-a388-26daed74fb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:55.295511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa8c01-a165-4fa2-a388-26daed74fb2f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:55.307358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1358/2668] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:45:55.312033 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9706f7-854a-446c-9922-ccca79153a3d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:55.323006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9706f7-854a-446c-9922-ccca79153a3d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:55.329791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:55.330456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:55.330760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:55.331039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] PATH_INFO: `/v3/projects/e705818597064414b9d882e0dd2071c2/users/52586f7b50964ec7ae104a04e9be3004/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:55.346732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e705818597064414b9d882e0dd2071c2, user_id=52586f7b50964ec7ae104a04e9be3004, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:55.349620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:55.363428 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1311/2669] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:45:55 2022] PUT /identity/v3/projects/e705818597064414b9d882e0dd2071c2/users/52586f7b50964ec7ae104a04e9be3004/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:45:55.368650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d32e-3580-4865-86c1-2f36bd9686d9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:55.368992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d32e-3580-4865-86c1-2f36bd9686d9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:55.369375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d32e-3580-4865-86c1-2f36bd9686d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:55.390070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4484d32e-3580-4865-86c1-2f36bd9686d9 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:55.391427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4484d32e-3580-4865-86c1-2f36bd9686d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:55.405526 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:45:55.461248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1359/2670] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:55.471666 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:55.473500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:55.558927 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecef4d-1406-4ee5-a801-e73ae2fb5215 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:55.563329 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53269b6-4191-405d-8afd-914725d2d341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:55.563948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53269b6-4191-405d-8afd-914725d2d341 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:55.564297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53269b6-4191-405d-8afd-914725d2d341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:55.564677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53269b6-4191-405d-8afd-914725d2d341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:55.576864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53269b6-4191-405d-8afd-914725d2d341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:55.579164 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53269b6-4191-405d-8afd-914725d2d341 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:55.589616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1312/2671] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:45:59.790670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b6be7-37e7-4509-b46e-cb31451c3d85 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:59.791231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b6be7-37e7-4509-b46e-cb31451c3d85 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:59.791573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b6be7-37e7-4509-b46e-cb31451c3d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:59.831518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-da6b6be7-37e7-4509-b46e-cb31451c3d85 None None] MFA Rules not processed for user `52586f7b50964ec7ae104a04e9be3004`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:45:59.831518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-da6b6be7-37e7-4509-b46e-cb31451c3d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:45:59.929923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1360/2672] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:45:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:45:59.935068 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53269b6-4191-405d-8afd-914725d2d341 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:45:59.946000 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53269b6-4191-405d-8afd-914725d2d341 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:45:59.951563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a5e8-3a66-4e49-b9b1-6d8f18f60b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:45:59.952187 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a5e8-3a66-4e49-b9b1-6d8f18f60b94 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:45:59.952863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a5e8-3a66-4e49-b9b1-6d8f18f60b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:45:59.953149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a5e8-3a66-4e49-b9b1-6d8f18f60b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:45:59.954422 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a5e8-3a66-4e49-b9b1-6d8f18f60b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:45:59.957153 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a5e8-3a66-4e49-b9b1-6d8f18f60b94 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:45:59.991497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1313/2673] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:45:59 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:45:59.995830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b6be7-37e7-4509-b46e-cb31451c3d85 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:00.011984 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b6be7-37e7-4509-b46e-cb31451c3d85 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:00.081649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cbf71-e3cc-4331-b98d-e45f0db9ad10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:00.083850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cbf71-e3cc-4331-b98d-e45f0db9ad10 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:00.084788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cbf71-e3cc-4331-b98d-e45f0db9ad10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:00.085178 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cbf71-e3cc-4331-b98d-e45f0db9ad10 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:00.086244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cbf71-e3cc-4331-b98d-e45f0db9ad10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:00.090235 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cbf71-e3cc-4331-b98d-e45f0db9ad10 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:00.137427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1361/2674] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:45:59 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:46:00.140369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a5e8-3a66-4e49-b9b1-6d8f18f60b94 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:00.151480 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a5e8-3a66-4e49-b9b1-6d8f18f60b94 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:00.156941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5fc9f-8abb-4b94-bcbf-8b199e0e5053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:00.157761 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5fc9f-8abb-4b94-bcbf-8b199e0e5053 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:00.158147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5fc9f-8abb-4b94-bcbf-8b199e0e5053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:00.158499 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5fc9f-8abb-4b94-bcbf-8b199e0e5053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:00.159284 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5fc9f-8abb-4b94-bcbf-8b199e0e5053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:00.161006 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5fc9f-8abb-4b94-bcbf-8b199e0e5053 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:00.173137 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1314/2675] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:46:00.177545 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cbf71-e3cc-4331-b98d-e45f0db9ad10 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:00.189167 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cbf71-e3cc-4331-b98d-e45f0db9ad10 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:00.194406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786e2ac-0aca-4ba6-8462-8a03fc2191fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:00.195203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786e2ac-0aca-4ba6-8462-8a03fc2191fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:00.195569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786e2ac-0aca-4ba6-8462-8a03fc2191fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:00.195854 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786e2ac-0aca-4ba6-8462-8a03fc2191fd admin admin] PATH_INFO: `/v3/projects/b3f5d1fadd61488cba568e09f880dcba/users/f16df18818914189a4890f25355e43e0/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:00.212810 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786e2ac-0aca-4ba6-8462-8a03fc2191fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f5d1fadd61488cba568e09f880dcba, user_id=f16df18818914189a4890f25355e43e0, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:00.215511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786e2ac-0aca-4ba6-8462-8a03fc2191fd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:00.229724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1362/2676] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:46:00 2022] PUT /identity/v3/projects/b3f5d1fadd61488cba568e09f880dcba/users/f16df18818914189a4890f25355e43e0/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:00.234336 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5fc9f-8abb-4b94-bcbf-8b199e0e5053 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:00.244863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5fc9f-8abb-4b94-bcbf-8b199e0e5053 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:00.285014 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:00.313876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:00.314627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:00.314893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:00.315262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:00.316234 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:00.317872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:00.328406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1315/2677] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:46:00.332850 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786e2ac-0aca-4ba6-8462-8a03fc2191fd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:00.344131 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786e2ac-0aca-4ba6-8462-8a03fc2191fd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:00.349669 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38c1d4-1cc3-4d14-85ac-91bf3b1aa1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:00.350361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38c1d4-1cc3-4d14-85ac-91bf3b1aa1c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:00.350649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38c1d4-1cc3-4d14-85ac-91bf3b1aa1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:00.350929 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38c1d4-1cc3-4d14-85ac-91bf3b1aa1c0 admin admin] PATH_INFO: `/v3/projects/b3f5d1fadd61488cba568e09f880dcba/users/f16df18818914189a4890f25355e43e0/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:00.352962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38c1d4-1cc3-4d14-85ac-91bf3b1aa1c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f5d1fadd61488cba568e09f880dcba, user_id=f16df18818914189a4890f25355e43e0, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:00.355664 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38c1d4-1cc3-4d14-85ac-91bf3b1aa1c0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:00.369755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1363/2678] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:46:00 2022] PUT /identity/v3/projects/b3f5d1fadd61488cba568e09f880dcba/users/f16df18818914189a4890f25355e43e0/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:02.510116 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:02.514089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:02.528957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc2491-8b2f-4029-8470-3807f3a67339 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:02.604773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:02.605461 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:02.605732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:02.606313 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] PATH_INFO: `/v3/users/9491acfc0a3547259f3d540c1aa78595` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:02.608326 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9491acfc0a3547259f3d540c1aa78595)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:02.610347 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:02.676679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9491acfc0a3547259f3d540c1aa78595'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:02.690452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9491acfc0a3547259f3d540c1aa78595'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:02.691523 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9491acfc0a3547259f3d540c1aa78595'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:02.715571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9491acfc0a3547259f3d540c1aa78595'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:02.716566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9491acfc0a3547259f3d540c1aa78595'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:02.735247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1316/2679] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:46:02 2022] DELETE /identity/v3/users/9491acfc0a3547259f3d540c1aa78595 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:02.736920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b3aa1-3802-4519-97ca-c30dbe41e09a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:02.737782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b3aa1-3802-4519-97ca-c30dbe41e09a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:02.738263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b3aa1-3802-4519-97ca-c30dbe41e09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:02.748852 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:02.767792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-417b3aa1-3802-4519-97ca-c30dbe41e09a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:02.769704 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-417b3aa1-3802-4519-97ca-c30dbe41e09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:02.898152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1364/2680] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:02.911040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:02.913644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:03.016546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7868d9-2599-4c8b-9732-38096901e5a3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:03.024982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e204b9-22fc-4ac9-8f72-280b821832d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:03.025805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e204b9-22fc-4ac9-8f72-280b821832d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:03.026196 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e204b9-22fc-4ac9-8f72-280b821832d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:03.026579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e204b9-22fc-4ac9-8f72-280b821832d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:03.040627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:03.048804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e204b9-22fc-4ac9-8f72-280b821832d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:03.050882 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e204b9-22fc-4ac9-8f72-280b821832d1 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:03.075438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1317/2681] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:03.928229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b3aa1-3802-4519-97ca-c30dbe41e09a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:03.930535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-417b3aa1-3802-4519-97ca-c30dbe41e09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:03.975493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b3aa1-3802-4519-97ca-c30dbe41e09a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:03.993738 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:03.995216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:03.995509 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:03.995792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] PATH_INFO: `/v3/projects/118b72df89d7441cb581f10755acc574` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:03.999000 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=118b72df89d7441cb581f10755acc574)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:04.009255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:04.113002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118b72df89d7441cb581f10755acc574', 'request_id': 'req-46b96b56-9c8c-4e78-866e-6920c21936e3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.120113 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118b72df89d7441cb581f10755acc574', 'request_id': 'req-46b96b56-9c8c-4e78-866e-6920c21936e3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.135086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118b72df89d7441cb581f10755acc574', 'request_id': 'req-46b96b56-9c8c-4e78-866e-6920c21936e3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.159029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] The token cache is being invalidate because project 118b72df89d7441cb581f10755acc574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:46:04.159332 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.159993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.161840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1365/2682] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:46:03 2022] DELETE /identity/v3/projects/118b72df89d7441cb581f10755acc574 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:04.165631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e204b9-22fc-4ac9-8f72-280b821832d1 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:04.176873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e204b9-22fc-4ac9-8f72-280b821832d1 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:04.252041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:04.252788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:04.253052 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:04.253366 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] PATH_INFO: `/v3/users/7c176182ac6f41f586b69ea58218dcd3` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:04.257637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c176182ac6f41f586b69ea58218dcd3)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:04.259575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:04.367620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c176182ac6f41f586b69ea58218dcd3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.393038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c176182ac6f41f586b69ea58218dcd3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.394415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c176182ac6f41f586b69ea58218dcd3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.414655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c176182ac6f41f586b69ea58218dcd3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.416915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-9c0b8555-91f0-47b7-8a8c-ab261c31aa5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c176182ac6f41f586b69ea58218dcd3'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:04.450495 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1318/2683] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:46:04 2022] DELETE /identity/v3/users/7c176182ac6f41f586b69ea58218dcd3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:05.064577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:05.067198 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:05.088716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96b56-9c8c-4e78-866e-6920c21936e3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:05.187993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:05.188840 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.189359 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.190002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] PATH_INFO: `/v3/projects/eff99f1ef72d422795776c3e324f7b7e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.191875 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff99f1ef72d422795776c3e324f7b7e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:05.194793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:05.237758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:05.261034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff99f1ef72d422795776c3e324f7b7e', 'request_id': 'req-10fe308a-f900-4e24-9fd8-c0a79244a271'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:05.270148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff99f1ef72d422795776c3e324f7b7e', 'request_id': 'req-10fe308a-f900-4e24-9fd8-c0a79244a271'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:05.283638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff99f1ef72d422795776c3e324f7b7e', 'request_id': 'req-10fe308a-f900-4e24-9fd8-c0a79244a271'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:05.301890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] The token cache is being invalidate because project eff99f1ef72d422795776c3e324f7b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:46:05.302109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:05.302789 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10fe308a-f900-4e24-9fd8-c0a79244a271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:05.304590 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1366/2684] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:46:05 2022] DELETE /identity/v3/projects/eff99f1ef72d422795776c3e324f7b7e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:05.309503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0970b3-3aee-4271-88db-5632fb6b24af None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.311078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0970b3-3aee-4271-88db-5632fb6b24af None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.311738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0970b3-3aee-4271-88db-5632fb6b24af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.338948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4e0970b3-3aee-4271-88db-5632fb6b24af None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:05.340402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4e0970b3-3aee-4271-88db-5632fb6b24af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:05.416375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1319/2685] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:46:05.426071 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acbc55-ae42-4cb6-905b-6b1f5531aac0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.426439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acbc55-ae42-4cb6-905b-6b1f5531aac0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.426641 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acbc55-ae42-4cb6-905b-6b1f5531aac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.450562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-25acbc55-ae42-4cb6-905b-6b1f5531aac0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:05.452133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-25acbc55-ae42-4cb6-905b-6b1f5531aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:05.472261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1367/2686] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:05.476497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0970b3-3aee-4271-88db-5632fb6b24af None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:05.488392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0970b3-3aee-4271-88db-5632fb6b24af None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:05.496517 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c65fd7-1a4e-4c12-9ad9-744645d7e866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:05.497479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65fd7-1a4e-4c12-9ad9-744645d7e866 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.498011 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65fd7-1a4e-4c12-9ad9-744645d7e866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.498697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65fd7-1a4e-4c12-9ad9-744645d7e866 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.500340 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c65fd7-1a4e-4c12-9ad9-744645d7e866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:46:05.500883 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c65fd7-1a4e-4c12-9ad9-744645d7e866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:05.502796 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c65fd7-1a4e-4c12-9ad9-744645d7e866 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:05.518721 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1320/2687] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 06:46:05.523336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acbc55-ae42-4cb6-905b-6b1f5531aac0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:05.536820 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acbc55-ae42-4cb6-905b-6b1f5531aac0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:05.541754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d0cf5-f039-4997-98be-c156db3eb4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:05.542495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d0cf5-f039-4997-98be-c156db3eb4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.543420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d0cf5-f039-4997-98be-c156db3eb4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.543420 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d0cf5-f039-4997-98be-c156db3eb4e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.544053 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d0cf5-f039-4997-98be-c156db3eb4e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:05.546599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d0cf5-f039-4997-98be-c156db3eb4e8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:05.585948 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c93b80-03ed-455e-9fa9-1ada3f266b60 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.586624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c93b80-03ed-455e-9fa9-1ada3f266b60 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.587310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c93b80-03ed-455e-9fa9-1ada3f266b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.588440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1368/2688] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:46:05 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:46:05.592959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d0cf5-f039-4997-98be-c156db3eb4e8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:05.598073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:05.626443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-59c93b80-03ed-455e-9fa9-1ada3f266b60 None None] MFA Rules not processed for user `f16df18818914189a4890f25355e43e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:05.627739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-59c93b80-03ed-455e-9fa9-1ada3f266b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:05.678748 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d0cf5-f039-4997-98be-c156db3eb4e8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:05.684491 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e48b34-0aeb-4ec2-b2a6-c14bdf5f468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:05.684780 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e48b34-0aeb-4ec2-b2a6-c14bdf5f468e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.685712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e48b34-0aeb-4ec2-b2a6-c14bdf5f468e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.686017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e48b34-0aeb-4ec2-b2a6-c14bdf5f468e admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.687016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e48b34-0aeb-4ec2-b2a6-c14bdf5f468e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:05.689253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e48b34-0aeb-4ec2-b2a6-c14bdf5f468e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:05.719172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1369/2689] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:46:05 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:46:05.722795 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e48b34-0aeb-4ec2-b2a6-c14bdf5f468e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:05.740150 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1321/2690] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:05.740599 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e48b34-0aeb-4ec2-b2a6-c14bdf5f468e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:05.746186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a3c44-268d-4fa3-9478-0b74149ea783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:05.747250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a3c44-268d-4fa3-9478-0b74149ea783 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.748042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a3c44-268d-4fa3-9478-0b74149ea783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.748325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a3c44-268d-4fa3-9478-0b74149ea783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.750038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c93b80-03ed-455e-9fa9-1ada3f266b60 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:05.751373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-59c93b80-03ed-455e-9fa9-1ada3f266b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:05.751566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a3c44-268d-4fa3-9478-0b74149ea783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:05.754858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a3c44-268d-4fa3-9478-0b74149ea783 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:05.773752 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1370/2691] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:46:05.775261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a3c44-268d-4fa3-9478-0b74149ea783 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:05.789589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a3c44-268d-4fa3-9478-0b74149ea783 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:05.797439 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd909-56b9-49b8-8498-f072937edd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:05.798552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd909-56b9-49b8-8498-f072937edd20 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.798700 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd909-56b9-49b8-8498-f072937edd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.798856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd909-56b9-49b8-8498-f072937edd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.799594 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdd909-56b9-49b8-8498-f072937edd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:05.801341 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdd909-56b9-49b8-8498-f072937edd20 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:05.837345 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1371/2692] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:46:05.839922 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd909-56b9-49b8-8498-f072937edd20 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:05.853761 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd909-56b9-49b8-8498-f072937edd20 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:05.859190 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420e32d-5970-463e-8345-2176d5c6fd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:05.860001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420e32d-5970-463e-8345-2176d5c6fd43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.860513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420e32d-5970-463e-8345-2176d5c6fd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.860655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c93b80-03ed-455e-9fa9-1ada3f266b60 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:05.860985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420e32d-5970-463e-8345-2176d5c6fd43 admin admin] PATH_INFO: `/v3/projects/e2b9fa0cf1944f988b737556a5b1cf9c/users/fdb6777374514f339fc4a86a10eee251/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.866646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:05.867280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.867555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.867835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.869713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:05.879467 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420e32d-5970-463e-8345-2176d5c6fd43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b9fa0cf1944f988b737556a5b1cf9c, user_id=fdb6777374514f339fc4a86a10eee251, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:05.882213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420e32d-5970-463e-8345-2176d5c6fd43 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:05.896996 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1372/2693] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:46:05 2022] PUT /identity/v3/projects/e2b9fa0cf1944f988b737556a5b1cf9c/users/fdb6777374514f339fc4a86a10eee251/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:05.907172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54288906-b5b4-4b58-a3d0-769b0d8e283f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:05.907595 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54288906-b5b4-4b58-a3d0-769b0d8e283f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:05.907982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54288906-b5b4-4b58-a3d0-769b0d8e283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:05.941226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-54288906-b5b4-4b58-a3d0-769b0d8e283f None None] MFA Rules not processed for user `fdb6777374514f339fc4a86a10eee251`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:05.942646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-54288906-b5b4-4b58-a3d0-769b0d8e283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:05.950914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:05.952583 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:05.964638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1322/2694] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:05.968475 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:06.041416 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1373/2695] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:46:06.045632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:06.058676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262446c-271a-4695-82c3-1ce668cd378d service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:06.086567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=100 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:06.136455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45864b-03c1-425d-84eb-21d95195d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:06.137372 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45864b-03c1-425d-84eb-21d95195d612 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:06.138627 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45864b-03c1-425d-84eb-21d95195d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:06.139059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45864b-03c1-425d-84eb-21d95195d612 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:06.140289 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45864b-03c1-425d-84eb-21d95195d612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:06.142664 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45864b-03c1-425d-84eb-21d95195d612 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:06.187322 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1323/2696] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:46:06 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:46:06.188133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54288906-b5b4-4b58-a3d0-769b0d8e283f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:06.222171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54288906-b5b4-4b58-a3d0-769b0d8e283f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:06.325091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee3fe1-143b-42c4-93d1-55e3bbac4920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:06.326683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee3fe1-143b-42c4-93d1-55e3bbac4920 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:06.327301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee3fe1-143b-42c4-93d1-55e3bbac4920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:06.327948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee3fe1-143b-42c4-93d1-55e3bbac4920 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:06.328665 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee3fe1-143b-42c4-93d1-55e3bbac4920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:06.331282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee3fe1-143b-42c4-93d1-55e3bbac4920 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:06.363532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1374/2697] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:46:06 2022] POST /identity/v3/users => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:46:06.367812 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45864b-03c1-425d-84eb-21d95195d612 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:06.381504 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45864b-03c1-425d-84eb-21d95195d612 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:06.387110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b819a-3f0f-4942-8da1-07a58d5c1184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:06.388520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b819a-3f0f-4942-8da1-07a58d5c1184 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:06.388905 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b819a-3f0f-4942-8da1-07a58d5c1184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:06.389231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b819a-3f0f-4942-8da1-07a58d5c1184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:06.390034 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b819a-3f0f-4942-8da1-07a58d5c1184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:06.391873 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b819a-3f0f-4942-8da1-07a58d5c1184 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:06.405247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1324/2698] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:46:06.409520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee3fe1-143b-42c4-93d1-55e3bbac4920 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:06.421045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee3fe1-143b-42c4-93d1-55e3bbac4920 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:06.429888 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f448c-0e8b-4484-a57a-0101c7ff67c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:06.430597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f448c-0e8b-4484-a57a-0101c7ff67c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:06.430880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f448c-0e8b-4484-a57a-0101c7ff67c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:06.431160 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f448c-0e8b-4484-a57a-0101c7ff67c8 admin admin] PATH_INFO: `/v3/projects/3bb4ab44453749249f984ef47408c2f5/users/e57fd27e5bb2417d8f0a2a07b9600f08/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:06.461214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f448c-0e8b-4484-a57a-0101c7ff67c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb4ab44453749249f984ef47408c2f5, user_id=e57fd27e5bb2417d8f0a2a07b9600f08, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:06.467023 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f448c-0e8b-4484-a57a-0101c7ff67c8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:06.497726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1375/2699] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:46:06 2022] PUT /identity/v3/projects/3bb4ab44453749249f984ef47408c2f5/users/e57fd27e5bb2417d8f0a2a07b9600f08/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:06.500208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b819a-3f0f-4942-8da1-07a58d5c1184 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:06.512582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b819a-3f0f-4942-8da1-07a58d5c1184 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:06.598427 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fcfea-c937-46c5-b435-0e305917be87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:06.599238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fcfea-c937-46c5-b435-0e305917be87 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:06.599597 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fcfea-c937-46c5-b435-0e305917be87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:06.599926 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fcfea-c937-46c5-b435-0e305917be87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:06.600762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fcfea-c937-46c5-b435-0e305917be87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:06.606270 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fcfea-c937-46c5-b435-0e305917be87 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:06.619808 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1325/2700] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:46:06.621104 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f448c-0e8b-4484-a57a-0101c7ff67c8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:06.633407 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f448c-0e8b-4484-a57a-0101c7ff67c8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:06.640118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:06.640903 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:06.641286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:06.641670 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] PATH_INFO: `/v3/projects/3bb4ab44453749249f984ef47408c2f5/users/e57fd27e5bb2417d8f0a2a07b9600f08/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:06.643837 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb4ab44453749249f984ef47408c2f5, user_id=e57fd27e5bb2417d8f0a2a07b9600f08, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:06.646511 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:06.660392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1376/2701] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:46:06 2022] PUT /identity/v3/projects/3bb4ab44453749249f984ef47408c2f5/users/e57fd27e5bb2417d8f0a2a07b9600f08/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:06.672098 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ba7fd-d948-48ba-8a65-81fbcdb422bd None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:06.672525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ba7fd-d948-48ba-8a65-81fbcdb422bd None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:06.673556 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ba7fd-d948-48ba-8a65-81fbcdb422bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:06.714612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8c3ba7fd-d948-48ba-8a65-81fbcdb422bd None None] MFA Rules not processed for user `e57fd27e5bb2417d8f0a2a07b9600f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:06.715983 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8c3ba7fd-d948-48ba-8a65-81fbcdb422bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:06.801469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1326/2702] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:06.878961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:06.880787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:06.948945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ede09-0d74-4e7e-b216-2206a91da787 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:06.953369 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe9d32-3891-46db-923b-761cac90fcca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:06.954011 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe9d32-3891-46db-923b-761cac90fcca service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:06.954293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe9d32-3891-46db-923b-761cac90fcca service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:06.954566 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe9d32-3891-46db-923b-761cac90fcca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:06.966950 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe9d32-3891-46db-923b-761cac90fcca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:06.968543 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe9d32-3891-46db-923b-761cac90fcca service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:06.978991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1377/2703] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:07.084631 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ba7fd-d948-48ba-8a65-81fbcdb422bd None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:07.097742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ba7fd-d948-48ba-8a65-81fbcdb422bd None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:07.102469 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb7c96-9558-4417-9a92-c956021602a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:07.103134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb7c96-9558-4417-9a92-c956021602a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:07.103998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb7c96-9558-4417-9a92-c956021602a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:07.103998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb7c96-9558-4417-9a92-c956021602a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:07.125073 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:07.179303 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb7c96-9558-4417-9a92-c956021602a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:07.180877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb7c96-9558-4417-9a92-c956021602a4 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:07.191701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1327/2704] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:07.219453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd4e2-68e4-43b8-8627-87b07e704a6a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:07.219992 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd4e2-68e4-43b8-8627-87b07e704a6a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:07.220208 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd4e2-68e4-43b8-8627-87b07e704a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:07.246822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3c0dd4e2-68e4-43b8-8627-87b07e704a6a None None] MFA Rules not processed for user `52586f7b50964ec7ae104a04e9be3004`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:07.248063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3c0dd4e2-68e4-43b8-8627-87b07e704a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:07.258401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:07.321001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1378/2705] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:07.511482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb7c96-9558-4417-9a92-c956021602a4 service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:07.584644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb7c96-9558-4417-9a92-c956021602a4 service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:07.590101 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:07.591791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:07.592811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:07.593287 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] PATH_INFO: `/v3/users/fdb6777374514f339fc4a86a10eee251` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:07.595144 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb6777374514f339fc4a86a10eee251)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:07.597911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:07.653661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb6777374514f339fc4a86a10eee251'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:07.663415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb6777374514f339fc4a86a10eee251'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:07.664226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb6777374514f339fc4a86a10eee251'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:07.677457 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb6777374514f339fc4a86a10eee251'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:07.678146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb6777374514f339fc4a86a10eee251'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:07.693462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1328/2706] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:46:07 2022] DELETE /identity/v3/users/fdb6777374514f339fc4a86a10eee251 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:07.698409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116f988-069f-4fc7-a74f-e959d28159ef None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:07.698600 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116f988-069f-4fc7-a74f-e959d28159ef None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:07.698858 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116f988-069f-4fc7-a74f-e959d28159ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:07.727943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b116f988-069f-4fc7-a74f-e959d28159ef None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:07.729313 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b116f988-069f-4fc7-a74f-e959d28159ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:07.873485 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1379/2707] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:07.880198 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:07.881815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:07.931345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:07.950595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f9094-b797-43af-bf2d-544022c9230e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:07.954310 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026367c-2457-4d4b-aa7f-ec6a5b9557c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:07.954936 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026367c-2457-4d4b-aa7f-ec6a5b9557c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:07.955220 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026367c-2457-4d4b-aa7f-ec6a5b9557c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:07.955497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026367c-2457-4d4b-aa7f-ec6a5b9557c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:07.967647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026367c-2457-4d4b-aa7f-ec6a5b9557c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:07.969325 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026367c-2457-4d4b-aa7f-ec6a5b9557c0 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:07.979707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1329/2708] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:08.260866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116f988-069f-4fc7-a74f-e959d28159ef None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:08.263643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b116f988-069f-4fc7-a74f-e959d28159ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:08.282513 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116f988-069f-4fc7-a74f-e959d28159ef None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:08.288923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:08.290405 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:08.291878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:08.292278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] PATH_INFO: `/v3/projects/e2b9fa0cf1944f988b737556a5b1cf9c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:08.295453 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b9fa0cf1944f988b737556a5b1cf9c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:08.299253 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:08.360054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b9fa0cf1944f988b737556a5b1cf9c', 'request_id': 'req-36e29cd0-e85c-4723-a8ed-912c9cb419ed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.366358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b9fa0cf1944f988b737556a5b1cf9c', 'request_id': 'req-36e29cd0-e85c-4723-a8ed-912c9cb419ed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.373974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b9fa0cf1944f988b737556a5b1cf9c', 'request_id': 'req-36e29cd0-e85c-4723-a8ed-912c9cb419ed'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.385504 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] The token cache is being invalidate because project e2b9fa0cf1944f988b737556a5b1cf9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:46:08.385798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.386446 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.388384 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1380/2709] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:46:08 2022] DELETE /identity/v3/projects/e2b9fa0cf1944f988b737556a5b1cf9c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:08.392311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026367c-2457-4d4b-aa7f-ec6a5b9557c0 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:08.404070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026367c-2457-4d4b-aa7f-ec6a5b9557c0 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:08.467051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:08.467815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:08.468099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:08.468381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] PATH_INFO: `/v3/users/e57fd27e5bb2417d8f0a2a07b9600f08` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:08.469789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e57fd27e5bb2417d8f0a2a07b9600f08)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:08.471793 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:08.522396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e57fd27e5bb2417d8f0a2a07b9600f08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.529823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e57fd27e5bb2417d8f0a2a07b9600f08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.530435 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e57fd27e5bb2417d8f0a2a07b9600f08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.542872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e57fd27e5bb2417d8f0a2a07b9600f08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.544812 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-4e7a2da7-749d-4def-87cb-d8b24f881ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e57fd27e5bb2417d8f0a2a07b9600f08'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:08.556913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1330/2710] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:46:08 2022] DELETE /identity/v3/users/e57fd27e5bb2417d8f0a2a07b9600f08 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:08.852536 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:08.854317 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:08.877081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e29cd0-e85c-4723-a8ed-912c9cb419ed admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:08.927881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:08.962514 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:08.963174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:08.963489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:08.963766 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] PATH_INFO: `/v3/projects/3bb4ab44453749249f984ef47408c2f5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:08.964957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb4ab44453749249f984ef47408c2f5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:08.967663 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:09.019462 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb4ab44453749249f984ef47408c2f5', 'request_id': 'req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:09.025577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb4ab44453749249f984ef47408c2f5', 'request_id': 'req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:09.034368 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb4ab44453749249f984ef47408c2f5', 'request_id': 'req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:09.046682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] The token cache is being invalidate because project 3bb4ab44453749249f984ef47408c2f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:46:09.046973 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:09.047647 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6d84fe1-42bb-4c2c-a762-150b20cf98f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:09.049180 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1381/2711] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:46:08 2022] DELETE /identity/v3/projects/3bb4ab44453749249f984ef47408c2f5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:09.054118 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861d21-aae5-47b7-be20-f5941ae6198b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.054415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861d21-aae5-47b7-be20-f5941ae6198b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.054729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74861d21-aae5-47b7-be20-f5941ae6198b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.076012 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-74861d21-aae5-47b7-be20-f5941ae6198b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:09.077349 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-74861d21-aae5-47b7-be20-f5941ae6198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:09.133067 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:09.146900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1331/2712] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:09.155564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55796afb-aa98-4060-b0ac-5c8358b7f412 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.155857 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55796afb-aa98-4060-b0ac-5c8358b7f412 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.156133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55796afb-aa98-4060-b0ac-5c8358b7f412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.176959 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-55796afb-aa98-4060-b0ac-5c8358b7f412 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:09.178207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-55796afb-aa98-4060-b0ac-5c8358b7f412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:09.187649 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1382/2713] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:09.192028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861d21-aae5-47b7-be20-f5941ae6198b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:09.202095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74861d21-aae5-47b7-be20-f5941ae6198b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:09.206904 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:09.207624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.207979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.208261 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.208963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:46:09.209269 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:09.210925 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:09.223905 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1332/2714] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:46:09.228340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55796afb-aa98-4060-b0ac-5c8358b7f412 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:09.238517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55796afb-aa98-4060-b0ac-5c8358b7f412 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:09.243277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da84e8-ff1f-4ae0-8857-39cc5404f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:09.243911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da84e8-ff1f-4ae0-8857-39cc5404f70b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.244189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da84e8-ff1f-4ae0-8857-39cc5404f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.244559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da84e8-ff1f-4ae0-8857-39cc5404f70b admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.245597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da84e8-ff1f-4ae0-8857-39cc5404f70b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:09.247775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da84e8-ff1f-4ae0-8857-39cc5404f70b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:09.281552 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1383/2715] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:46:09 2022] POST /identity/v3/projects => generated 359 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:46:09.288070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:09.301055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d44e9-5f31-47a7-9ca4-76cb0aec244c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:09.365286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74001e2d-f30d-463b-b923-e75e988e2b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:09.366046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74001e2d-f30d-463b-b923-e75e988e2b1b admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.366335 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74001e2d-f30d-463b-b923-e75e988e2b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.366610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74001e2d-f30d-463b-b923-e75e988e2b1b admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.367280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74001e2d-f30d-463b-b923-e75e988e2b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:09.369222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74001e2d-f30d-463b-b923-e75e988e2b1b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:09.393972 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1333/2716] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:46:09 2022] POST /identity/v3/users => generated 280 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:46:09.398421 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da84e8-ff1f-4ae0-8857-39cc5404f70b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:09.413503 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da84e8-ff1f-4ae0-8857-39cc5404f70b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:09.418254 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7ae8f-a197-45a5-b549-b8dc8ee92e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:09.418926 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7ae8f-a197-45a5-b549-b8dc8ee92e19 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.419215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7ae8f-a197-45a5-b549-b8dc8ee92e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.419488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7ae8f-a197-45a5-b549-b8dc8ee92e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.420312 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7ae8f-a197-45a5-b549-b8dc8ee92e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:09.422055 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7ae8f-a197-45a5-b549-b8dc8ee92e19 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:09.441349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1384/2717] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:46:09.445334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74001e2d-f30d-463b-b923-e75e988e2b1b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:09.458917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74001e2d-f30d-463b-b923-e75e988e2b1b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:09.464111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2946a7-5dae-469a-9b8b-1ba7e36c6420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:09.464828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2946a7-5dae-469a-9b8b-1ba7e36c6420 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.465076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2946a7-5dae-469a-9b8b-1ba7e36c6420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.465390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2946a7-5dae-469a-9b8b-1ba7e36c6420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.468938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2946a7-5dae-469a-9b8b-1ba7e36c6420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:09.470596 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2946a7-5dae-469a-9b8b-1ba7e36c6420 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:09.484695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1334/2718] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:46:09.489106 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7ae8f-a197-45a5-b549-b8dc8ee92e19 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:09.502304 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7ae8f-a197-45a5-b549-b8dc8ee92e19 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:09.507449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:09.508153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.508586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.508733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] PATH_INFO: `/v3/projects/6c687c516d034af2b04db826f630f3df/users/ecf52794c823488b85d24aad47659c3a/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.527470 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c687c516d034af2b04db826f630f3df, user_id=ecf52794c823488b85d24aad47659c3a, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:09.530250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:09.546431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1385/2719] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:46:09 2022] PUT /identity/v3/projects/6c687c516d034af2b04db826f630f3df/users/ecf52794c823488b85d24aad47659c3a/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:46:09.556730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35573-a496-423c-a50a-37af841cc59a None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.557259 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35573-a496-423c-a50a-37af841cc59a None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.557605 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35573-a496-423c-a50a-37af841cc59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.593401 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-9fd35573-a496-423c-a50a-37af841cc59a None None] MFA Rules not processed for user `ecf52794c823488b85d24aad47659c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:09.594852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-9fd35573-a496-423c-a50a-37af841cc59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:09.678491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1335/2720] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:09 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:09.724148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:09.726806 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:09.852798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea567e-b2cb-47c3-8a1c-9b7392e45e76 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:09.858839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664ed9e-dc43-4aa7-8e10-91584c22f7ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:09.859791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664ed9e-dc43-4aa7-8e10-91584c22f7ce service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:09.859980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664ed9e-dc43-4aa7-8e10-91584c22f7ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:09.860362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664ed9e-dc43-4aa7-8e10-91584c22f7ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:09.873689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664ed9e-dc43-4aa7-8e10-91584c22f7ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:09.875338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664ed9e-dc43-4aa7-8e10-91584c22f7ce service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:09.900559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1386/2721] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:09 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:09.910384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35573-a496-423c-a50a-37af841cc59a None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:10.017918 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:10.023565 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35573-a496-423c-a50a-37af841cc59a None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:10.030219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:10.031257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:10.031893 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:10.032240 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] PATH_INFO: `/v3/users/ecf52794c823488b85d24aad47659c3a` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:10.033542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf52794c823488b85d24aad47659c3a)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:10.035538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:10.102092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf52794c823488b85d24aad47659c3a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.116373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf52794c823488b85d24aad47659c3a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.117451 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf52794c823488b85d24aad47659c3a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.135544 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf52794c823488b85d24aad47659c3a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.136830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf52794c823488b85d24aad47659c3a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.153935 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1336/2722] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:46:09 2022] DELETE /identity/v3/users/ecf52794c823488b85d24aad47659c3a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:10.158758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2ac51-71b9-4fb5-b65a-24c460609e59 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:10.159440 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2ac51-71b9-4fb5-b65a-24c460609e59 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:10.160656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2ac51-71b9-4fb5-b65a-24c460609e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:10.186411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a1e2ac51-71b9-4fb5-b65a-24c460609e59 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:10.187672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a1e2ac51-71b9-4fb5-b65a-24c460609e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:10.197777 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:10.271031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1387/2723] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:10.281584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:10.283026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:10.348485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa651-1a3f-4f24-b77e-dd7c9b136b70 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:10.352203 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fda813-d62f-4310-a271-c47e938026e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:10.352835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fda813-d62f-4310-a271-c47e938026e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:10.353117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fda813-d62f-4310-a271-c47e938026e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:10.353433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fda813-d62f-4310-a271-c47e938026e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:10.364113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fda813-d62f-4310-a271-c47e938026e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:10.365578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fda813-d62f-4310-a271-c47e938026e1 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:10.376474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1337/2724] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:10.685246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2ac51-71b9-4fb5-b65a-24c460609e59 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:10.686722 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a1e2ac51-71b9-4fb5-b65a-24c460609e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:10.699197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2ac51-71b9-4fb5-b65a-24c460609e59 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:10.704532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:10.704978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:10.705263 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:10.705718 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] PATH_INFO: `/v3/projects/6c687c516d034af2b04db826f630f3df` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:10.707238 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c687c516d034af2b04db826f630f3df)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:10.709186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:10.769758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c687c516d034af2b04db826f630f3df', 'request_id': 'req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.776593 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c687c516d034af2b04db826f630f3df', 'request_id': 'req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.785830 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c687c516d034af2b04db826f630f3df', 'request_id': 'req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.802003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] The token cache is being invalidate because project 6c687c516d034af2b04db826f630f3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:46:10.802291 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.802952 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-411c8bd7-3ba3-43f0-a38a-19fdac74fdf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:10.807898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1388/2725] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:46:10 2022] DELETE /identity/v3/projects/6c687c516d034af2b04db826f630f3df => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:10.809678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbf4c7-0c81-452e-93a0-ed57d130bbf6 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:10.810283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbf4c7-0c81-452e-93a0-ed57d130bbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:10.810283 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbf4c7-0c81-452e-93a0-ed57d130bbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:10.832783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-bfdbf4c7-0c81-452e-93a0-ed57d130bbf6 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:10.834027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-bfdbf4c7-0c81-452e-93a0-ed57d130bbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:10.900010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1338/2726] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:10.910577 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff2f8-280b-45db-b1a1-7635c5f1b339 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:10.910871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff2f8-280b-45db-b1a1-7635c5f1b339 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:10.911153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff2f8-280b-45db-b1a1-7635c5f1b339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:10.931716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7c3ff2f8-280b-45db-b1a1-7635c5f1b339 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:10.932853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7c3ff2f8-280b-45db-b1a1-7635c5f1b339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:10.941727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1389/2727] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:10.946194 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbf4c7-0c81-452e-93a0-ed57d130bbf6 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:10.957076 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbf4c7-0c81-452e-93a0-ed57d130bbf6 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:10.962550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:10.963162 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:10.963446 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:10.963722 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:10.964602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:46:10.964906 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:10.968037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:10.980545 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1339/2728] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:46:10.985287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ff2f8-280b-45db-b1a1-7635c5f1b339 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:10.994690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ff2f8-280b-45db-b1a1-7635c5f1b339 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:10.999744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a190ca-66c0-45f7-b3cd-79019f75e80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:11.000372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a190ca-66c0-45f7-b3cd-79019f75e80f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:11.000655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a190ca-66c0-45f7-b3cd-79019f75e80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:11.000933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a190ca-66c0-45f7-b3cd-79019f75e80f admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:11.003920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a190ca-66c0-45f7-b3cd-79019f75e80f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:11.006092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a190ca-66c0-45f7-b3cd-79019f75e80f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:11.039733 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1390/2729] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:46:10 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:46:11.041254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:11.049235 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:11.053683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696dda7-68f3-4405-abc9-fdbd88b2ea17 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:11.116756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba2a4c-012a-44e6-aea7-1f2e8d55cfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:11.117410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba2a4c-012a-44e6-aea7-1f2e8d55cfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:11.117700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba2a4c-012a-44e6-aea7-1f2e8d55cfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:11.117980 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba2a4c-012a-44e6-aea7-1f2e8d55cfb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:11.118665 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba2a4c-012a-44e6-aea7-1f2e8d55cfb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:11.120581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba2a4c-012a-44e6-aea7-1f2e8d55cfb4 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:11.142634 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1340/2730] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:46:11 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:46:11.147216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a190ca-66c0-45f7-b3cd-79019f75e80f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:11.159244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a190ca-66c0-45f7-b3cd-79019f75e80f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:11.163921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098bf47-afb3-4dd1-8e92-110638d941fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:11.164908 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098bf47-afb3-4dd1-8e92-110638d941fd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:11.165194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098bf47-afb3-4dd1-8e92-110638d941fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:11.165510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098bf47-afb3-4dd1-8e92-110638d941fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:11.166237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098bf47-afb3-4dd1-8e92-110638d941fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:11.167822 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098bf47-afb3-4dd1-8e92-110638d941fd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:11.178109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1391/2731] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:46:11.182406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba2a4c-012a-44e6-aea7-1f2e8d55cfb4 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:11.193762 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba2a4c-012a-44e6-aea7-1f2e8d55cfb4 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:11.198603 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bbd93-c470-4d1d-a696-f52a1d1148c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:11.199448 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbd93-c470-4d1d-a696-f52a1d1148c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:11.199733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbd93-c470-4d1d-a696-f52a1d1148c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:11.200046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbd93-c470-4d1d-a696-f52a1d1148c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:11.201356 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bbd93-c470-4d1d-a696-f52a1d1148c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:11.202958 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bbd93-c470-4d1d-a696-f52a1d1148c6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:11.215002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1341/2732] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:46:11.220137 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098bf47-afb3-4dd1-8e92-110638d941fd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:11.230316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098bf47-afb3-4dd1-8e92-110638d941fd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:11.235499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:11.236162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:11.236444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:11.236721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] PATH_INFO: `/v3/projects/69b915e2497348609f7f18d803df3d51/users/e686ee04ee7a4491a7993c4afd4250c3/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:11.254482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b915e2497348609f7f18d803df3d51, user_id=e686ee04ee7a4491a7993c4afd4250c3, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:11.257165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:11.270482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1392/2733] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:46:11 2022] PUT /identity/v3/projects/69b915e2497348609f7f18d803df3d51/users/e686ee04ee7a4491a7993c4afd4250c3/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:11.280978 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d71034-0a5d-4206-af75-242b8e16e4f0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:11.281272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d71034-0a5d-4206-af75-242b8e16e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:11.281593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d71034-0a5d-4206-af75-242b8e16e4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:11.310674 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-52d71034-0a5d-4206-af75-242b8e16e4f0 None None] MFA Rules not processed for user `e686ee04ee7a4491a7993c4afd4250c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:11.311890 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-52d71034-0a5d-4206-af75-242b8e16e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:11.379149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1342/2734] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:11.391162 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:11.392834 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:11.428465 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:11.454086 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b0e49-989a-49a0-9390-c0f448cc6ac2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:11.457737 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898576f-0cc7-427b-9e78-21089fd5e1f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '649334bb7ed04bd69eb971fc070d96e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:11.458356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898576f-0cc7-427b-9e78-21089fd5e1f3 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:11.458639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898576f-0cc7-427b-9e78-21089fd5e1f3 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:11.458919 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898576f-0cc7-427b-9e78-21089fd5e1f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:11.467697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898576f-0cc7-427b-9e78-21089fd5e1f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:11.469135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898576f-0cc7-427b-9e78-21089fd5e1f3 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:11.478096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1393/2735] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:21.376231 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:21.474061 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=11 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:46.775088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d71034-0a5d-4206-af75-242b8e16e4f0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:46.776500 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-52d71034-0a5d-4206-af75-242b8e16e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:46.838286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d71034-0a5d-4206-af75-242b8e16e4f0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:46.842183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabcb34-4345-4494-8547-24a9503d06d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:46.842908 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabcb34-4345-4494-8547-24a9503d06d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:46.843265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabcb34-4345-4494-8547-24a9503d06d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:46.843616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabcb34-4345-4494-8547-24a9503d06d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:46.845777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-3dabcb34-4345-4494-8547-24a9503d06d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:46.928642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabcb34-4345-4494-8547-24a9503d06d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:46.930141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabcb34-4345-4494-8547-24a9503d06d4 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:46.940783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1343/2736] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:46:49.152151 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898576f-0cc7-427b-9e78-21089fd5e1f3 service glance] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:49.217042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898576f-0cc7-427b-9e78-21089fd5e1f3 service glance] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:49.221697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:49.222339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:49.222617 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:49.222890 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] PATH_INFO: `/v3/users/e686ee04ee7a4491a7993c4afd4250c3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:49.224760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e686ee04ee7a4491a7993c4afd4250c3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:49.226693 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:49.298084 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e686ee04ee7a4491a7993c4afd4250c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:49.309745 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e686ee04ee7a4491a7993c4afd4250c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:49.324744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e686ee04ee7a4491a7993c4afd4250c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:49.325499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e686ee04ee7a4491a7993c4afd4250c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:49.342951 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e686ee04ee7a4491a7993c4afd4250c3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:49.350134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1394/2737] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:46:49 2022] DELETE /identity/v3/users/e686ee04ee7a4491a7993c4afd4250c3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:49.355176 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6fd3-2e54-45e6-89ba-239eb3a6ac10 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:49.355719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6fd3-2e54-45e6-89ba-239eb3a6ac10 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:49.355719 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6fd3-2e54-45e6-89ba-239eb3a6ac10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:49.382420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4eeb6fd3-2e54-45e6-89ba-239eb3a6ac10 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:49.384089 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4eeb6fd3-2e54-45e6-89ba-239eb3a6ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:49.463913 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1344/2738] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:49.473755 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:49.475428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:49.538303 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3a315-2ebd-4f8a-aecf-72bb8b22d8ba admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:49.541115 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3425f-57f0-4fd4-80bc-d1ec408fd324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:49.541776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3425f-57f0-4fd4-80bc-d1ec408fd324 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:49.542052 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3425f-57f0-4fd4-80bc-d1ec408fd324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:49.542320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3425f-57f0-4fd4-80bc-d1ec408fd324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:49.552744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3425f-57f0-4fd4-80bc-d1ec408fd324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:49.554234 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3425f-57f0-4fd4-80bc-d1ec408fd324 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:49.566081 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1395/2739] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:46:49.990967 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6fd3-2e54-45e6-89ba-239eb3a6ac10 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:49.992726 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4eeb6fd3-2e54-45e6-89ba-239eb3a6ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:50.008317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6fd3-2e54-45e6-89ba-239eb3a6ac10 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:50.014777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:50.015537 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.016308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.016661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] PATH_INFO: `/v3/projects/69b915e2497348609f7f18d803df3d51` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.018385 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b915e2497348609f7f18d803df3d51)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:50.020940 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:50.070030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:50.073368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b915e2497348609f7f18d803df3d51', 'request_id': 'req-f50053fa-2016-4cc4-a7b9-ad2d482beefb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:50.081826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b915e2497348609f7f18d803df3d51', 'request_id': 'req-f50053fa-2016-4cc4-a7b9-ad2d482beefb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:50.089416 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b915e2497348609f7f18d803df3d51', 'request_id': 'req-f50053fa-2016-4cc4-a7b9-ad2d482beefb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:50.100830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] The token cache is being invalidate because project 69b915e2497348609f7f18d803df3d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:46:50.101100 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:50.101725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-f50053fa-2016-4cc4-a7b9-ad2d482beefb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:46:50.103395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1345/2740] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:46:49 2022] DELETE /identity/v3/projects/69b915e2497348609f7f18d803df3d51 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:46:50.108417 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff9d8-b52e-4b47-84e3-129b2cb3468b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.108760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff9d8-b52e-4b47-84e3-129b2cb3468b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.109054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff9d8-b52e-4b47-84e3-129b2cb3468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.131097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-77bff9d8-b52e-4b47-84e3-129b2cb3468b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:50.132315 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-77bff9d8-b52e-4b47-84e3-129b2cb3468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:50.149723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:46:50.196019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1396/2741] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:50.205509 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f9d9-7855-4ffc-9e4a-afd08db99fa3 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.205875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f9d9-7855-4ffc-9e4a-afd08db99fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.206229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f9d9-7855-4ffc-9e4a-afd08db99fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.226745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-c521f9d9-7855-4ffc-9e4a-afd08db99fa3 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:50.227957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c521f9d9-7855-4ffc-9e4a-afd08db99fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:50.236307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1346/2742] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:46:50.240687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bff9d8-b52e-4b47-84e3-129b2cb3468b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:50.250302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bff9d8-b52e-4b47-84e3-129b2cb3468b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:50.255319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:50.255937 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.256219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.256496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.257195 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:46:50.257539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:50.259119 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:50.274378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1397/2743] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:46:50.275987 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521f9d9-7855-4ffc-9e4a-afd08db99fa3 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:50.286149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521f9d9-7855-4ffc-9e4a-afd08db99fa3 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:50.290852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea09a8-be83-4ca6-8bca-831647e6e7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:50.291934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea09a8-be83-4ca6-8bca-831647e6e7cc admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.292277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea09a8-be83-4ca6-8bca-831647e6e7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.292645 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea09a8-be83-4ca6-8bca-831647e6e7cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.293655 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea09a8-be83-4ca6-8bca-831647e6e7cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:50.296829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea09a8-be83-4ca6-8bca-831647e6e7cc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:50.329602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1347/2744] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:46:50 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:46:50.334002 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:50.343947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebd8a2-4069-405d-896b-8371bec26da7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:50.405335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3dd9e7-6396-4459-99a7-b0a57127a7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:50.405880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dd9e7-6396-4459-99a7-b0a57127a7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.406158 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dd9e7-6396-4459-99a7-b0a57127a7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.406429 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dd9e7-6396-4459-99a7-b0a57127a7b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.407112 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dd9e7-6396-4459-99a7-b0a57127a7b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:50.408989 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dd9e7-6396-4459-99a7-b0a57127a7b1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:50.429562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1398/2745] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:46:50 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:46:50.433947 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea09a8-be83-4ca6-8bca-831647e6e7cc admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:50.445323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea09a8-be83-4ca6-8bca-831647e6e7cc admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:50.449852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07dafe-08b0-4076-befc-7394491f6d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:50.450456 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07dafe-08b0-4076-befc-7394491f6d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.450734 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07dafe-08b0-4076-befc-7394491f6d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.451009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07dafe-08b0-4076-befc-7394491f6d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.451698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07dafe-08b0-4076-befc-7394491f6d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:50.453374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07dafe-08b0-4076-befc-7394491f6d7e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:50.463959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1348/2746] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:46:50.468187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3dd9e7-6396-4459-99a7-b0a57127a7b1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:50.478391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3dd9e7-6396-4459-99a7-b0a57127a7b1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:50.483201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d5b22-e41a-4169-b1ca-65b3c816aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:50.483802 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5b22-e41a-4169-b1ca-65b3c816aea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.484080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5b22-e41a-4169-b1ca-65b3c816aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.484351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5b22-e41a-4169-b1ca-65b3c816aea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.485065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d5b22-e41a-4169-b1ca-65b3c816aea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:50.486692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d5b22-e41a-4169-b1ca-65b3c816aea5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:50.497365 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1399/2747] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:46:50.501405 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07dafe-08b0-4076-befc-7394491f6d7e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:50.511771 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07dafe-08b0-4076-befc-7394491f6d7e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:50.516578 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:50.517334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.517620 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.517896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] PATH_INFO: `/v3/projects/b16cc00d564349c69091fc4a00e13cba/users/ac6d5f4767b84720aa399215595d64bc/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.532974 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16cc00d564349c69091fc4a00e13cba, user_id=ac6d5f4767b84720aa399215595d64bc, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:50.535644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:50.548533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1349/2748] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:46:50 2022] PUT /identity/v3/projects/b16cc00d564349c69091fc4a00e13cba/users/ac6d5f4767b84720aa399215595d64bc/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:46:50.558904 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b93ac3-e7d7-441d-bdb6-c5746d2b7d20 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.559196 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b93ac3-e7d7-441d-bdb6-c5746d2b7d20 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.559476 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b93ac3-e7d7-441d-bdb6-c5746d2b7d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.588042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-57b93ac3-e7d7-441d-bdb6-c5746d2b7d20 None None] MFA Rules not processed for user `ac6d5f4767b84720aa399215595d64bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:46:50.589242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-57b93ac3-e7d7-441d-bdb6-c5746d2b7d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:50.652292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1400/2749] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:46:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:46:50.663979 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:46:50.665696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:46:50.724600 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c57bd-64ed-47c8-9be2-ee18fef46e3b admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:46:50.728443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde42e5-9d39-469d-bd1b-ca1263e9b343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:46:50.729090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde42e5-9d39-469d-bd1b-ca1263e9b343 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:46:50.729404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde42e5-9d39-469d-bd1b-ca1263e9b343 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:46:50.729683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde42e5-9d39-469d-bd1b-ca1263e9b343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:46:50.737941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde42e5-9d39-469d-bd1b-ca1263e9b343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:46:50.739930 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde42e5-9d39-469d-bd1b-ca1263e9b343 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:46:50.748679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1350/2750] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:46:50 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:47:00.649264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:47:00.744613 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:47:26.865938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b93ac3-e7d7-441d-bdb6-c5746d2b7d20 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:26.930101 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b93ac3-e7d7-441d-bdb6-c5746d2b7d20 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:26.935499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:26.936136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:26.936455 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:26.936721 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] PATH_INFO: `/v3/users/ac6d5f4767b84720aa399215595d64bc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:26.937896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac6d5f4767b84720aa399215595d64bc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:26.939798 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:26.987223 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac6d5f4767b84720aa399215595d64bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:26.994631 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac6d5f4767b84720aa399215595d64bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.001474 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac6d5f4767b84720aa399215595d64bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.002098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac6d5f4767b84720aa399215595d64bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.014540 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac6d5f4767b84720aa399215595d64bc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.020265 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1401/2751] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:47:26 2022] DELETE /identity/v3/users/ac6d5f4767b84720aa399215595d64bc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:47:27.024851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31e9f-716a-477c-a1e7-b8c2e6004ffb None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:27.025413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31e9f-716a-477c-a1e7-b8c2e6004ffb None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:27.025567 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e31e9f-716a-477c-a1e7-b8c2e6004ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:27.044907 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-85e31e9f-716a-477c-a1e7-b8c2e6004ffb None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:47:27.046147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-85e31e9f-716a-477c-a1e7-b8c2e6004ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:27.111957 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1351/2752] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:47:27.122171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:27.123598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:27.183310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1397c4-e5d8-4402-84fa-adea9be2caaa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:27.186976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de015cb-cebc-4ee4-ba4a-66e2cf328f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:27.187591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de015cb-cebc-4ee4-ba4a-66e2cf328f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:27.188034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de015cb-cebc-4ee4-ba4a-66e2cf328f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:27.188316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de015cb-cebc-4ee4-ba4a-66e2cf328f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:27.198880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de015cb-cebc-4ee4-ba4a-66e2cf328f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:27.199793 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de015cb-cebc-4ee4-ba4a-66e2cf328f9e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:27.209207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1402/2753] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:47:27.477048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e31e9f-716a-477c-a1e7-b8c2e6004ffb None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:27.479160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-85e31e9f-716a-477c-a1e7-b8c2e6004ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:27.491046 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e31e9f-716a-477c-a1e7-b8c2e6004ffb None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:27.495994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:27.496642 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:27.496921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:27.497195 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] PATH_INFO: `/v3/projects/b16cc00d564349c69091fc4a00e13cba` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:27.498413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16cc00d564349c69091fc4a00e13cba)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:27.500308 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:27.547637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16cc00d564349c69091fc4a00e13cba', 'request_id': 'req-d121aef3-1160-4147-b545-fdef05604ee1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.555433 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16cc00d564349c69091fc4a00e13cba', 'request_id': 'req-d121aef3-1160-4147-b545-fdef05604ee1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.561147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16cc00d564349c69091fc4a00e13cba', 'request_id': 'req-d121aef3-1160-4147-b545-fdef05604ee1'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.572421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] The token cache is being invalidate because project b16cc00d564349c69091fc4a00e13cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:47:27.572710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.573379 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:27.574888 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1352/2754] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:47:27 2022] DELETE /identity/v3/projects/b16cc00d564349c69091fc4a00e13cba => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:47:33.036461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de015cb-cebc-4ee4-ba4a-66e2cf328f9e service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:33.046702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de015cb-cebc-4ee4-ba4a-66e2cf328f9e service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:33.067548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:47:33.101807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:33.102654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:33.103492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:33.103871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:33.105686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:33.184391 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:33.185871 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:33.198747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1403/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:47:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:47:34.391652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:34.393250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:34.406215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121aef3-1160-4147-b545-fdef05604ee1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:34.409745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:34.410360 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:34.410641 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:34.410910 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:34.412638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:34.427113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:34.428536 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:34.443036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1353/2756] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:47:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:47:39.896264 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:39.905404 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4013975-89bb-4a43-9923-52182de65d35 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:39.909427 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:39.910135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:39.910489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:39.911344 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:39.912845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:39.927636 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:39.929201 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:39.939351 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1404/2757] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:47:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:47:44.438337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:47:45.391683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:45.400334 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ceb69-4a91-4476-ba4a-2b4800040b4b service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:45.404170 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:45.404770 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:45.405048 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:45.405375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:45.407099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:45.470601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:45.472091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:45.481884 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1354/2758] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:47:48.611889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:48.613557 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:48.704216 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f138a3-ce78-45d5-9fb0-059e702a7dcf service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:48.709261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:48.709899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:48.710175 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:48.710449 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:48.712360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:48.724746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:48.726348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:48.737042 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1405/2759] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:47:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:47:51.876082 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:51.877751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:51.891163 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312b108-1755-4668-9f06-c4d9d26b89d6 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:51.896439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:51.897086 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:51.897395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:51.897669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] PATH_INFO: `/v3/users/13ca4be686f84c279f87016c20129833` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:51.898818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ca4be686f84c279f87016c20129833)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:51.900964 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:51.950229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ca4be686f84c279f87016c20129833'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:51.957134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ca4be686f84c279f87016c20129833'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:51.957784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ca4be686f84c279f87016c20129833'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:51.970317 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ca4be686f84c279f87016c20129833'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:51.970965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ca4be686f84c279f87016c20129833'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:51.983278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1355/2760] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:47:51 2022] DELETE /identity/v3/users/13ca4be686f84c279f87016c20129833 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:47:52.160122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:52.173772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ae64e-9838-4053-a3c7-de2b79e16807 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:52.212894 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=100 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:47:52.233203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:52.233879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:52.234166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:52.234441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] PATH_INFO: `/v3/projects/f16eee0fb716452e8c760a27b5724d91` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:52.235597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16eee0fb716452e8c760a27b5724d91)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:52.237571 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:52.283586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16eee0fb716452e8c760a27b5724d91', 'request_id': 'req-657cc2a4-e43d-48a4-b2dd-14445eb7966c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.289063 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16eee0fb716452e8c760a27b5724d91', 'request_id': 'req-657cc2a4-e43d-48a4-b2dd-14445eb7966c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.296716 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16eee0fb716452e8c760a27b5724d91', 'request_id': 'req-657cc2a4-e43d-48a4-b2dd-14445eb7966c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.307832 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] The token cache is being invalidate because project f16eee0fb716452e8c760a27b5724d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:47:52.308122 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.308758 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.310298 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1406/2761] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:47:52 2022] DELETE /identity/v3/projects/f16eee0fb716452e8c760a27b5724d91 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:47:52.314323 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:52.315968 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:52.329256 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7084e09-2dc3-4b3f-84c0-e7d9b9c50c92 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:52.387900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:52.388481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:52.388788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:52.389051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] PATH_INFO: `/v3/users/f37a9c5000c54dada98baebc43684446` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:52.390277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f37a9c5000c54dada98baebc43684446)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:52.392185 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:52.397439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:47:52.439571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f37a9c5000c54dada98baebc43684446'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.447292 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f37a9c5000c54dada98baebc43684446'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.448679 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f37a9c5000c54dada98baebc43684446'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.460818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f37a9c5000c54dada98baebc43684446'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.461518 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f37a9c5000c54dada98baebc43684446'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.475099 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1356/2762] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:47:52 2022] DELETE /identity/v3/users/f37a9c5000c54dada98baebc43684446 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:47:52.650443 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:52.652555 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:52.665328 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cc2a4-e43d-48a4-b2dd-14445eb7966c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:52.725250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:52.725917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:52.726199 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:52.726473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] PATH_INFO: `/v3/projects/a771358cf916419593e60477ec58b5fa` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:52.727625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a771358cf916419593e60477ec58b5fa)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:52.729779 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:52.776547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a771358cf916419593e60477ec58b5fa', 'request_id': 'req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.782046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a771358cf916419593e60477ec58b5fa', 'request_id': 'req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.790007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a771358cf916419593e60477ec58b5fa', 'request_id': 'req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.798462 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:47:52.802626 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] The token cache is being invalidate because project a771358cf916419593e60477ec58b5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:47:52.803126 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.804046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-685c9d9c-cfd8-4d56-8e6c-c75cd276f67c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:47:52.805781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1407/2763] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:47:52 2022] DELETE /identity/v3/projects/a771358cf916419593e60477ec58b5fa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:47:59.232546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:47:59.234698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:59.323659 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05e76f-52ec-4217-9984-2460b6037228 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:47:59.400563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:47:59.402058 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:47:59.403010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:47:59.403387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:47:59.405258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:47:59.420002 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:47:59.421675 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:47:59.434747 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1357/2764] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:47:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:48:02.230683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365f79-ca06-4b42-a18e-da6cc87194b4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:48:02.231226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365f79-ca06-4b42-a18e-da6cc87194b4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:48:02.231355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e365f79-ca06-4b42-a18e-da6cc87194b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:48:02.250437 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0e365f79-ca06-4b42-a18e-da6cc87194b4 None None] MFA Rules not processed for user `a87b79d20f0f4194a99fc342c9b14aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:48:02.251625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0e365f79-ca06-4b42-a18e-da6cc87194b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:48:02.311450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1408/2765] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:48:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:48:02.621879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:48:02.623288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:48:02.633371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:48:02.681685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3c28a-7245-4d7a-84a5-8866df2eeefb service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:48:02.685402 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:48:02.686016 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:48:02.686299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:48:02.686574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:48:02.688375 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:48:02.698069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:48:02.699636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:48:02.707566 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1358/2766] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:48:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:48:03.034867 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e365f79-ca06-4b42-a18e-da6cc87194b4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:48:03.036260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0e365f79-ca06-4b42-a18e-da6cc87194b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:48:03.107981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e365f79-ca06-4b42-a18e-da6cc87194b4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:48:03.113102 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:48:03.113763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:48:03.114045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:48:03.114323 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:48:03.116096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:48:03.128616 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:48:03.130225 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:48:03.140487 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1409/2767] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:48:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:48:12.605091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:48:12.606546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:48:12.623572 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281520ef-b7c5-4780-92be-ab7a653953d7 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:48:12.628946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:48:12.629592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:48:12.629876 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:48:12.630152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:48:12.631905 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:48:12.644829 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:48:12.646431 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:48:12.657387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1359/2768] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:48:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:48:13.135813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=57 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:48:15.758481 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:48:15.771282 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adb9d2-f9f1-4a8d-9074-bc39724ece81 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:48:15.777045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f558687eefe4a26beb7d69151221660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:48:15.777698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:48:15.777980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:48:15.778255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:48:15.780174 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:48:15.793624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:48:15.795203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:48:15.807389 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1410/2769] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:48:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:48:22.653434 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=28 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:48:25.802018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=6 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:08.084226 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:08.085773 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:08.143828 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615b16b-c8f9-4788-beaf-2257b245e433 service nova] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:08.147707 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:08.148330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:08.148617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:08.148899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:08.150737 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:08.161821 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:08.163249 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:08.174477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1360/2770] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:18.167998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=26 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:23.725495 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:23.726881 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:23.738865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f0e3a-5d40-4ab0-9fc7-cf07a73a1fc4 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:23.742377 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:23.742978 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:23.743256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:23.743527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:23.745176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:23.805729 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:23.807185 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:23.817488 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1411/2771] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:27.014962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.028305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ca9df-2313-46f0-85e7-d253a2f65c6d service cinder] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.034337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.034942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.035227 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.035503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.037338 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:27.052258 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.054139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.079551 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.080152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1361/2772] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:27.081171 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:27.098027 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c260b-e557-4e9b-8751-0a4e0c816012 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.102909 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.103561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.103845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.104129 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] PATH_INFO: `/v3/users/a87b79d20f0f4194a99fc342c9b14aa2` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.106715 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87b79d20f0f4194a99fc342c9b14aa2)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.108626 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.159097 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87b79d20f0f4194a99fc342c9b14aa2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.165999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87b79d20f0f4194a99fc342c9b14aa2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.173673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87b79d20f0f4194a99fc342c9b14aa2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.174348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87b79d20f0f4194a99fc342c9b14aa2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.186496 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ae216700-7923-4e7f-8d01-2a0d5749c3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87b79d20f0f4194a99fc342c9b14aa2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.193764 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1412/2773] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:27 2022] DELETE /identity/v3/users/a87b79d20f0f4194a99fc342c9b14aa2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:49:27.370555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.382552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036e874-a7d2-475a-b110-698acfa29223 service placement] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.442663 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.443359 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.443643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.443920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] PATH_INFO: `/v3/projects/f0f65f082a2547f79ac218e318bee171` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.445072 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f65f082a2547f79ac218e318bee171)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.446998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.493616 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f65f082a2547f79ac218e318bee171', 'request_id': 'req-e2f354df-ae0c-4434-beda-cc2fb03e44cc'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.501470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f65f082a2547f79ac218e318bee171', 'request_id': 'req-e2f354df-ae0c-4434-beda-cc2fb03e44cc'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.507017 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f65f082a2547f79ac218e318bee171', 'request_id': 'req-e2f354df-ae0c-4434-beda-cc2fb03e44cc'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.518772 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] The token cache is being invalidate because project f0f65f082a2547f79ac218e318bee171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:27.519062 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.519701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-e2f354df-ae0c-4434-beda-cc2fb03e44cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:27.521280 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1362/2774] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:27 2022] DELETE /identity/v3/projects/f0f65f082a2547f79ac218e318bee171 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:27.526401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f684e65-6919-42bc-952d-89c74f23ba43 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.526714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f684e65-6919-42bc-952d-89c74f23ba43 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.526981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f684e65-6919-42bc-952d-89c74f23ba43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.547203 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4f684e65-6919-42bc-952d-89c74f23ba43 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:27.548340 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4f684e65-6919-42bc-952d-89c74f23ba43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:27.605547 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1413/2775] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:27.614290 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0d90-51ed-4567-ac47-fa270f898bf7 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.614676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0d90-51ed-4567-ac47-fa270f898bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.614982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0d90-51ed-4567-ac47-fa270f898bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.635384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-358f0d90-51ed-4567-ac47-fa270f898bf7 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:27.636584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-358f0d90-51ed-4567-ac47-fa270f898bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:27.644648 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1363/2776] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:27.648912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f684e65-6919-42bc-952d-89c74f23ba43 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.658699 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f684e65-6919-42bc-952d-89c74f23ba43 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.663302 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.663910 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.664193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.664468 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.665165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:49:27.665510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.667092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.668546 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:27.681596 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1414/2777] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:49:27.685851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f0d90-51ed-4567-ac47-fa270f898bf7 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.695872 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f0d90-51ed-4567-ac47-fa270f898bf7 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.700561 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567a683-4bb2-415c-bfe5-2dc72ac58794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.701162 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567a683-4bb2-415c-bfe5-2dc72ac58794 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.701481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567a683-4bb2-415c-bfe5-2dc72ac58794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.701756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567a683-4bb2-415c-bfe5-2dc72ac58794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.702421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567a683-4bb2-415c-bfe5-2dc72ac58794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.704311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567a683-4bb2-415c-bfe5-2dc72ac58794 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.740396 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1364/2778] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:27 2022] POST /identity/v3/projects => generated 401 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:27.744464 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.755383 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293aa399-4b16-4c7b-8ad5-6f88257261bb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.813103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752efd7-a629-49e1-83ec-2e28b3dd7602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.813740 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752efd7-a629-49e1-83ec-2e28b3dd7602 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.814003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752efd7-a629-49e1-83ec-2e28b3dd7602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.814278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752efd7-a629-49e1-83ec-2e28b3dd7602 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.814940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752efd7-a629-49e1-83ec-2e28b3dd7602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.817128 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752efd7-a629-49e1-83ec-2e28b3dd7602 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.845360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1415/2779] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:27 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:27.849609 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567a683-4bb2-415c-bfe5-2dc72ac58794 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.860760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567a683-4bb2-415c-bfe5-2dc72ac58794 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.865555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9156a7b-e753-48ca-a833-d4d9925ef4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.866173 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9156a7b-e753-48ca-a833-d4d9925ef4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.866580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9156a7b-e753-48ca-a833-d4d9925ef4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.866860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9156a7b-e753-48ca-a833-d4d9925ef4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.867586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9156a7b-e753-48ca-a833-d4d9925ef4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.869181 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9156a7b-e753-48ca-a833-d4d9925ef4e8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.880593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1365/2780] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:27.884622 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752efd7-a629-49e1-83ec-2e28b3dd7602 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.895079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752efd7-a629-49e1-83ec-2e28b3dd7602 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.899791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b3ede-cb10-40df-8b66-edbd95d91b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.900406 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b3ede-cb10-40df-8b66-edbd95d91b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.900692 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b3ede-cb10-40df-8b66-edbd95d91b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.900966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b3ede-cb10-40df-8b66-edbd95d91b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.901687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b3ede-cb10-40df-8b66-edbd95d91b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.903299 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b3ede-cb10-40df-8b66-edbd95d91b0f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.916998 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1416/2781] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:27.921158 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9156a7b-e753-48ca-a833-d4d9925ef4e8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:27.937468 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9156a7b-e753-48ca-a833-d4d9925ef4e8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:27.942466 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:27.943128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.943415 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.943695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] PATH_INFO: `/v3/projects/a4e1a21d17db4be59e478342d02a9c20/users/efda4d371b684a83a3809f204dc96ac5/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:27.959550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e1a21d17db4be59e478342d02a9c20, user_id=efda4d371b684a83a3809f204dc96ac5, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:27.962534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:27.975278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1366/2782] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:27 2022] PUT /identity/v3/projects/a4e1a21d17db4be59e478342d02a9c20/users/efda4d371b684a83a3809f204dc96ac5/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:49:27.980353 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbb6f-5634-4513-aa62-6281ccf114d0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:27.980625 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbb6f-5634-4513-aa62-6281ccf114d0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:27.980896 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbb6f-5634-4513-aa62-6281ccf114d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:28.000640 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-809fbb6f-5634-4513-aa62-6281ccf114d0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:28.001876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-809fbb6f-5634-4513-aa62-6281ccf114d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:28.060211 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1417/2783] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:28.070792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:28.072522 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:28.079938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:28.130167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24e754-e099-40bb-a7ab-b92cc2e6e5e2 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:28.133815 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87691183-8155-4e93-9448-166837425519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:28.134429 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87691183-8155-4e93-9448-166837425519 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:28.134713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87691183-8155-4e93-9448-166837425519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:28.134989 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87691183-8155-4e93-9448-166837425519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:28.144492 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87691183-8155-4e93-9448-166837425519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:28.145944 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87691183-8155-4e93-9448-166837425519 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:28.155367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1367/2784] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:28.778818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a06766-83fa-4dc4-9f02-25c539524779 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:28.779388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a06766-83fa-4dc4-9f02-25c539524779 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:28.779388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a06766-83fa-4dc4-9f02-25c539524779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:28.809482 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b1a06766-83fa-4dc4-9f02-25c539524779 None None] MFA Rules not processed for user `efda4d371b684a83a3809f204dc96ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:28.811220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b1a06766-83fa-4dc4-9f02-25c539524779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:28.877911 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1418/2785] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:28.882615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87691183-8155-4e93-9448-166837425519 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:28.892248 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87691183-8155-4e93-9448-166837425519 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:28.896717 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6164e5-598f-40e6-810a-8931afd276a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:28.897346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6164e5-598f-40e6-810a-8931afd276a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:28.897629 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6164e5-598f-40e6-810a-8931afd276a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:28.897900 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6164e5-598f-40e6-810a-8931afd276a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:28.898573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6164e5-598f-40e6-810a-8931afd276a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:28.900445 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6164e5-598f-40e6-810a-8931afd276a8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:28.928812 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1368/2786] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:28 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:28.932673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a06766-83fa-4dc4-9f02-25c539524779 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:28.941017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:28.947668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a06766-83fa-4dc4-9f02-25c539524779 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:29.005843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbbf77-01ba-447d-b883-e343df5e0f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:29.006490 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbbf77-01ba-447d-b883-e343df5e0f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:29.006775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbbf77-01ba-447d-b883-e343df5e0f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:29.007051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbbf77-01ba-447d-b883-e343df5e0f23 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:29.007728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbbf77-01ba-447d-b883-e343df5e0f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:29.009728 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbbf77-01ba-447d-b883-e343df5e0f23 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:29.031727 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1419/2787] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:28 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:29.035690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6164e5-598f-40e6-810a-8931afd276a8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:29.046070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6164e5-598f-40e6-810a-8931afd276a8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:29.050625 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05100b9e-2103-497d-9f30-5221e9bc7175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:29.051223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100b9e-2103-497d-9f30-5221e9bc7175 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:29.051503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100b9e-2103-497d-9f30-5221e9bc7175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:29.051775 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100b9e-2103-497d-9f30-5221e9bc7175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:29.052459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05100b9e-2103-497d-9f30-5221e9bc7175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:29.054208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05100b9e-2103-497d-9f30-5221e9bc7175 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:29.066459 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1369/2788] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:29.070362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbbf77-01ba-447d-b883-e343df5e0f23 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:29.081532 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbbf77-01ba-447d-b883-e343df5e0f23 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:29.086098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2963fc-e3c2-4c3b-bbe2-ad3f1e185ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:29.086754 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2963fc-e3c2-4c3b-bbe2-ad3f1e185ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:29.087031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2963fc-e3c2-4c3b-bbe2-ad3f1e185ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:29.087301 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2963fc-e3c2-4c3b-bbe2-ad3f1e185ca3 admin admin] PATH_INFO: `/v3/projects/4d535711e2494d1b87e591f54be1a711/users/5805874216454fccb81688b9862302dc/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:29.103714 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2963fc-e3c2-4c3b-bbe2-ad3f1e185ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d535711e2494d1b87e591f54be1a711, user_id=5805874216454fccb81688b9862302dc, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:29.107408 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2963fc-e3c2-4c3b-bbe2-ad3f1e185ca3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:29.120749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1420/2789] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:29 2022] PUT /identity/v3/projects/4d535711e2494d1b87e591f54be1a711/users/5805874216454fccb81688b9862302dc/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:29.125037 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05100b9e-2103-497d-9f30-5221e9bc7175 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:29.135013 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05100b9e-2103-497d-9f30-5221e9bc7175 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:29.194121 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef35d80-0792-438f-a625-5efaad47b25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:29.194661 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35d80-0792-438f-a625-5efaad47b25e admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:29.194946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35d80-0792-438f-a625-5efaad47b25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:29.195219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35d80-0792-438f-a625-5efaad47b25e admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:29.195911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef35d80-0792-438f-a625-5efaad47b25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:29.197521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef35d80-0792-438f-a625-5efaad47b25e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:29.208027 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1370/2790] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:29.212260 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2963fc-e3c2-4c3b-bbe2-ad3f1e185ca3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:29.223096 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2963fc-e3c2-4c3b-bbe2-ad3f1e185ca3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:29.227845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56284f-6ca7-4e74-9fac-8d968d077733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:29.228494 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56284f-6ca7-4e74-9fac-8d968d077733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:29.228776 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56284f-6ca7-4e74-9fac-8d968d077733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:29.229044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56284f-6ca7-4e74-9fac-8d968d077733 admin admin] PATH_INFO: `/v3/projects/4d535711e2494d1b87e591f54be1a711/users/5805874216454fccb81688b9862302dc/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:29.231001 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56284f-6ca7-4e74-9fac-8d968d077733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d535711e2494d1b87e591f54be1a711, user_id=5805874216454fccb81688b9862302dc, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:29.233626 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56284f-6ca7-4e74-9fac-8d968d077733 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:29.243991 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1421/2791] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:29 2022] PUT /identity/v3/projects/4d535711e2494d1b87e591f54be1a711/users/5805874216454fccb81688b9862302dc/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:29.888678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c528b-26d4-48f7-8f5f-c179010e6e70 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:29.889059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c528b-26d4-48f7-8f5f-c179010e6e70 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:29.889475 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c528b-26d4-48f7-8f5f-c179010e6e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:29.917644 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f45c528b-26d4-48f7-8f5f-c179010e6e70 None None] MFA Rules not processed for user `5805874216454fccb81688b9862302dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:29.918849 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f45c528b-26d4-48f7-8f5f-c179010e6e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:29.983611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:29.996130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1371/2792] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:31.034056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56284f-6ca7-4e74-9fac-8d968d077733 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:31.097899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56284f-6ca7-4e74-9fac-8d968d077733 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:31.102444 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:31.103073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:31.103355 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:31.103632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] PATH_INFO: `/v3/users/efda4d371b684a83a3809f204dc96ac5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:31.104787 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efda4d371b684a83a3809f204dc96ac5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:31.106726 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:31.152661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efda4d371b684a83a3809f204dc96ac5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.159955 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efda4d371b684a83a3809f204dc96ac5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.161624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:31.168176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efda4d371b684a83a3809f204dc96ac5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.169280 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efda4d371b684a83a3809f204dc96ac5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.182039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efda4d371b684a83a3809f204dc96ac5'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.188380 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1422/2793] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:31 2022] DELETE /identity/v3/users/efda4d371b684a83a3809f204dc96ac5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:31.370117 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c528b-26d4-48f7-8f5f-c179010e6e70 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:31.381701 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c528b-26d4-48f7-8f5f-c179010e6e70 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:31.439976 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:31.440800 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:31.441080 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:31.441404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] PATH_INFO: `/v3/projects/a4e1a21d17db4be59e478342d02a9c20` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:31.442579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e1a21d17db4be59e478342d02a9c20)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:31.444584 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:31.490955 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e1a21d17db4be59e478342d02a9c20', 'request_id': 'req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.497474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e1a21d17db4be59e478342d02a9c20', 'request_id': 'req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.503879 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e1a21d17db4be59e478342d02a9c20', 'request_id': 'req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.526798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] The token cache is being invalidate because project a4e1a21d17db4be59e478342d02a9c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:31.527091 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.527756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.529286 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1372/2794] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:31 2022] DELETE /identity/v3/projects/a4e1a21d17db4be59e478342d02a9c20 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:31.534100 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:31.535561 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:31.548349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c694c03-2071-44bc-89cb-3f96d71d7a96 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:31.603730 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:31.604373 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:31.604658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:31.604933 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] PATH_INFO: `/v3/users/5805874216454fccb81688b9862302dc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:31.606152 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5805874216454fccb81688b9862302dc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:31.608056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:31.653114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5805874216454fccb81688b9862302dc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.660650 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5805874216454fccb81688b9862302dc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.667762 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5805874216454fccb81688b9862302dc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.668411 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5805874216454fccb81688b9862302dc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.680258 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f9ee8c9a-11ee-40d4-a378-8d63fb325bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5805874216454fccb81688b9862302dc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.686088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1423/2795] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:31 2022] DELETE /identity/v3/users/5805874216454fccb81688b9862302dc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:49:31.858615 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:31.861001 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:31.876714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c65141-72dc-45e0-abda-6ed5d09b6fcb admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:31.936152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:31.936797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:31.937594 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:31.937875 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] PATH_INFO: `/v3/projects/4d535711e2494d1b87e591f54be1a711` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:31.939041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d535711e2494d1b87e591f54be1a711)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:31.941252 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:31.972511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:31.989728 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d535711e2494d1b87e591f54be1a711', 'request_id': 'req-bba544aa-7351-424f-bcf4-e8ed49052f63'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:31.996750 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d535711e2494d1b87e591f54be1a711', 'request_id': 'req-bba544aa-7351-424f-bcf4-e8ed49052f63'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:32.002758 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d535711e2494d1b87e591f54be1a711', 'request_id': 'req-bba544aa-7351-424f-bcf4-e8ed49052f63'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:32.013538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] The token cache is being invalidate because project 4d535711e2494d1b87e591f54be1a711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:32.013826 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:32.014452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-bba544aa-7351-424f-bcf4-e8ed49052f63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:32.016965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1373/2796] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:31 2022] DELETE /identity/v3/projects/4d535711e2494d1b87e591f54be1a711 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:32.022342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafadbb-160b-4f91-a3a3-e730e3085f04 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.022639 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafadbb-160b-4f91-a3a3-e730e3085f04 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.022917 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafadbb-160b-4f91-a3a3-e730e3085f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.042841 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3cafadbb-160b-4f91-a3a3-e730e3085f04 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:32.044035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3cafadbb-160b-4f91-a3a3-e730e3085f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:32.090869 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:32.105103 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1424/2797] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:32.112969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808adeb-13bc-4b9c-b703-17f00570bf84 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.113263 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808adeb-13bc-4b9c-b703-17f00570bf84 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.113573 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808adeb-13bc-4b9c-b703-17f00570bf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.132823 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-f808adeb-13bc-4b9c-b703-17f00570bf84 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:32.134065 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-f808adeb-13bc-4b9c-b703-17f00570bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:32.142732 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1374/2798] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:32.147019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafadbb-160b-4f91-a3a3-e730e3085f04 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.157654 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafadbb-160b-4f91-a3a3-e730e3085f04 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.162349 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.162957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.163269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.163564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.164262 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:49:32.164567 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.166187 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.177986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1425/2799] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:49:32.182210 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808adeb-13bc-4b9c-b703-17f00570bf84 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.191700 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808adeb-13bc-4b9c-b703-17f00570bf84 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.196282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ca637-127d-49ac-b6a9-abe9b5969cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.196886 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca637-127d-49ac-b6a9-abe9b5969cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.197172 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca637-127d-49ac-b6a9-abe9b5969cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.197482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca637-127d-49ac-b6a9-abe9b5969cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.198178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ca637-127d-49ac-b6a9-abe9b5969cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.200057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ca637-127d-49ac-b6a9-abe9b5969cbd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.227298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1375/2800] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:32 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:32.231339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.241591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250adc7-43d9-4e23-a84e-114a7f7c180a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.297433 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b5316-f67a-4cc3-b6d4-d91a610557d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.298051 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b5316-f67a-4cc3-b6d4-d91a610557d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.298339 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b5316-f67a-4cc3-b6d4-d91a610557d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.298614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b5316-f67a-4cc3-b6d4-d91a610557d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.299273 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b5316-f67a-4cc3-b6d4-d91a610557d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.301165 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b5316-f67a-4cc3-b6d4-d91a610557d8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.322218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1426/2801] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:32 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:32.326346 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ca637-127d-49ac-b6a9-abe9b5969cbd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.337555 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ca637-127d-49ac-b6a9-abe9b5969cbd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.342140 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c08918-401e-4d70-88f5-5d89b17a0dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.342751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08918-401e-4d70-88f5-5d89b17a0dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.343036 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08918-401e-4d70-88f5-5d89b17a0dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.344215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08918-401e-4d70-88f5-5d89b17a0dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.344916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c08918-401e-4d70-88f5-5d89b17a0dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.346552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c08918-401e-4d70-88f5-5d89b17a0dc7 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.356262 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1376/2802] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:32.360070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b5316-f67a-4cc3-b6d4-d91a610557d8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.369819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b5316-f67a-4cc3-b6d4-d91a610557d8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.374363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3eec3-7637-4a4e-8f68-1c961755f6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.374974 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3eec3-7637-4a4e-8f68-1c961755f6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.375244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3eec3-7637-4a4e-8f68-1c961755f6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.375517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3eec3-7637-4a4e-8f68-1c961755f6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.376197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3eec3-7637-4a4e-8f68-1c961755f6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.378747 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3eec3-7637-4a4e-8f68-1c961755f6f8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.390378 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1427/2803] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:32.394363 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c08918-401e-4d70-88f5-5d89b17a0dc7 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.404470 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c08918-401e-4d70-88f5-5d89b17a0dc7 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.409923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd6e7c-d086-4152-98b8-e25385701215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.410574 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd6e7c-d086-4152-98b8-e25385701215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.410857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd6e7c-d086-4152-98b8-e25385701215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.411130 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd6e7c-d086-4152-98b8-e25385701215 admin admin] PATH_INFO: `/v3/projects/b30614691f844edf8ff077f77141677b/users/34eebdd0e079412c88fd71f4309f9e56/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.425781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd6e7c-d086-4152-98b8-e25385701215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30614691f844edf8ff077f77141677b, user_id=34eebdd0e079412c88fd71f4309f9e56, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.428511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd6e7c-d086-4152-98b8-e25385701215 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.446738 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1377/2804] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:32 2022] PUT /identity/v3/projects/b30614691f844edf8ff077f77141677b/users/34eebdd0e079412c88fd71f4309f9e56/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:49:32.512148 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9fd7d-5728-4b7e-862e-af3d046c59b7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.512436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9fd7d-5728-4b7e-862e-af3d046c59b7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.512711 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9fd7d-5728-4b7e-862e-af3d046c59b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.540990 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-80e9fd7d-5728-4b7e-862e-af3d046c59b7 None None] MFA Rules not processed for user `34eebdd0e079412c88fd71f4309f9e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:32.542257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-80e9fd7d-5728-4b7e-862e-af3d046c59b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:32.607098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1428/2805] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:32.611951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd6e7c-d086-4152-98b8-e25385701215 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.672751 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd6e7c-d086-4152-98b8-e25385701215 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.677576 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f59a0-0808-458e-a043-d95555b4dbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.678224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f59a0-0808-458e-a043-d95555b4dbe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.678511 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f59a0-0808-458e-a043-d95555b4dbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.678788 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f59a0-0808-458e-a043-d95555b4dbe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.679453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f59a0-0808-458e-a043-d95555b4dbe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.681377 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f59a0-0808-458e-a043-d95555b4dbe1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.709538 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1378/2806] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:32 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:32.713479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9fd7d-5728-4b7e-862e-af3d046c59b7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.724953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9fd7d-5728-4b7e-862e-af3d046c59b7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.770783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:32.785117 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1c2c0-5ba7-45a8-81b2-d6128a82dc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.785783 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c2c0-5ba7-45a8-81b2-d6128a82dc69 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.786069 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c2c0-5ba7-45a8-81b2-d6128a82dc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.786360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c2c0-5ba7-45a8-81b2-d6128a82dc69 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.787077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1c2c0-5ba7-45a8-81b2-d6128a82dc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.788976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1c2c0-5ba7-45a8-81b2-d6128a82dc69 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.811479 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1429/2807] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:32 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:32.816381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f59a0-0808-458e-a043-d95555b4dbe1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.826776 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f59a0-0808-458e-a043-d95555b4dbe1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.831590 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc175e1-161c-4600-b8b0-4111921d21c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.832197 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc175e1-161c-4600-b8b0-4111921d21c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.832479 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc175e1-161c-4600-b8b0-4111921d21c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.832757 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc175e1-161c-4600-b8b0-4111921d21c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.833483 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc175e1-161c-4600-b8b0-4111921d21c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.835069 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc175e1-161c-4600-b8b0-4111921d21c2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.846561 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1379/2808] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:49:32.850658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1c2c0-5ba7-45a8-81b2-d6128a82dc69 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.861218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1c2c0-5ba7-45a8-81b2-d6128a82dc69 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.865938 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fb235-007c-4596-abbd-77e42bba5925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.866592 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb235-007c-4596-abbd-77e42bba5925 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.866877 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb235-007c-4596-abbd-77e42bba5925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.867153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb235-007c-4596-abbd-77e42bba5925 admin admin] PATH_INFO: `/v3/projects/fdeccef2b33a43cc949adcccc9e68071/users/e809094c82ed45baa9b59adf0718f43a/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.882539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fb235-007c-4596-abbd-77e42bba5925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdeccef2b33a43cc949adcccc9e68071, user_id=e809094c82ed45baa9b59adf0718f43a, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.885279 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fb235-007c-4596-abbd-77e42bba5925 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.898346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1430/2809] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:32 2022] PUT /identity/v3/projects/fdeccef2b33a43cc949adcccc9e68071/users/e809094c82ed45baa9b59adf0718f43a/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:49:32.902236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc175e1-161c-4600-b8b0-4111921d21c2 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:32.912686 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc175e1-161c-4600-b8b0-4111921d21c2 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:32.936443 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:32.970998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71ef43-eaa9-4a91-a5c8-3996c85b2a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:32.971542 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71ef43-eaa9-4a91-a5c8-3996c85b2a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:32.971833 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71ef43-eaa9-4a91-a5c8-3996c85b2a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:32.972111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71ef43-eaa9-4a91-a5c8-3996c85b2a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:32.972824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71ef43-eaa9-4a91-a5c8-3996c85b2a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:32.974448 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71ef43-eaa9-4a91-a5c8-3996c85b2a44 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:32.984799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1380/2810] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:32.989261 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fb235-007c-4596-abbd-77e42bba5925 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:33.000666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fb235-007c-4596-abbd-77e42bba5925 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:33.005250 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:33.005935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:33.006218 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:33.006492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] PATH_INFO: `/v3/projects/fdeccef2b33a43cc949adcccc9e68071/users/e809094c82ed45baa9b59adf0718f43a/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:33.008576 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdeccef2b33a43cc949adcccc9e68071, user_id=e809094c82ed45baa9b59adf0718f43a, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:33.011269 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:33.021499 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1431/2811] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:32 2022] PUT /identity/v3/projects/fdeccef2b33a43cc949adcccc9e68071/users/e809094c82ed45baa9b59adf0718f43a/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:33.031369 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74f120-a471-46c4-8724-6198492abeca None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:33.031658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74f120-a471-46c4-8724-6198492abeca None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:33.031934 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74f120-a471-46c4-8724-6198492abeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:33.059494 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8f74f120-a471-46c4-8724-6198492abeca None None] MFA Rules not processed for user `e809094c82ed45baa9b59adf0718f43a`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:33.060685 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8f74f120-a471-46c4-8724-6198492abeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:33.128559 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1381/2812] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:33.140804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:33.152045 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:33.211697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae3904-477f-4f4d-a2c9-175a2846f23c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:33.215331 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c395d9-2b62-4a52-8adb-53bdea56eb36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:33.215939 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c395d9-2b62-4a52-8adb-53bdea56eb36 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:33.216219 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c395d9-2b62-4a52-8adb-53bdea56eb36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:33.216493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c395d9-2b62-4a52-8adb-53bdea56eb36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:33.225945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c395d9-2b62-4a52-8adb-53bdea56eb36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:33.227372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c395d9-2b62-4a52-8adb-53bdea56eb36 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:33.237085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1432/2813] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 06:49:33.325797 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74f120-a471-46c4-8724-6198492abeca None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:33.336963 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74f120-a471-46c4-8724-6198492abeca None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:33.340670 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1c279-7756-4430-8783-dc42f2b77b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:33.341358 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1c279-7756-4430-8783-dc42f2b77b61 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:33.341646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1c279-7756-4430-8783-dc42f2b77b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:33.341924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1c279-7756-4430-8783-dc42f2b77b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:33.398858 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1c279-7756-4430-8783-dc42f2b77b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:33.400277 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1c279-7756-4430-8783-dc42f2b77b61 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:33.409167 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1382/2814] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:43.232618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=48,keystone:INSERT=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:43.405015 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=79 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:43.660658 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c395d9-2b62-4a52-8adb-53bdea56eb36 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:43.722804 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c395d9-2b62-4a52-8adb-53bdea56eb36 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:43.727379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:43.728046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:43.728329 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:43.728607 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] PATH_INFO: `/v3/users/34eebdd0e079412c88fd71f4309f9e56` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:43.729788 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34eebdd0e079412c88fd71f4309f9e56)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:43.731687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:43.775949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34eebdd0e079412c88fd71f4309f9e56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:43.783237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34eebdd0e079412c88fd71f4309f9e56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:43.790310 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34eebdd0e079412c88fd71f4309f9e56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:43.791037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34eebdd0e079412c88fd71f4309f9e56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:43.803878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34eebdd0e079412c88fd71f4309f9e56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:43.809866 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1433/2815] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:43 2022] DELETE /identity/v3/users/34eebdd0e079412c88fd71f4309f9e56 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:43.815010 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808255-7ba2-4baf-8d7b-1e16e4d1a041 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:43.815302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808255-7ba2-4baf-8d7b-1e16e4d1a041 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:43.815837 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808255-7ba2-4baf-8d7b-1e16e4d1a041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:43.834753 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-eb808255-7ba2-4baf-8d7b-1e16e4d1a041 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:43.835951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eb808255-7ba2-4baf-8d7b-1e16e4d1a041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:43.893733 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1383/2816] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:49:43.903522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:43.904928 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:43.963956 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15242fec-0ece-47a7-8254-250be7a0cdb7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:43.967529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9626c3-5ce6-4e58-8bed-eb7e3e534905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:43.968749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9626c3-5ce6-4e58-8bed-eb7e3e534905 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:43.969058 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9626c3-5ce6-4e58-8bed-eb7e3e534905 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:43.969361 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9626c3-5ce6-4e58-8bed-eb7e3e534905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:43.978390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9626c3-5ce6-4e58-8bed-eb7e3e534905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:43.979808 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9626c3-5ce6-4e58-8bed-eb7e3e534905 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:43.990220 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1434/2817] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:44.260985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb808255-7ba2-4baf-8d7b-1e16e4d1a041 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:44.262472 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eb808255-7ba2-4baf-8d7b-1e16e4d1a041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:44.275390 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb808255-7ba2-4baf-8d7b-1e16e4d1a041 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:44.280943 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:44.281704 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:44.281991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:44.282267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] PATH_INFO: `/v3/projects/b30614691f844edf8ff077f77141677b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:44.283452 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30614691f844edf8ff077f77141677b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:44.285387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:44.331417 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30614691f844edf8ff077f77141677b', 'request_id': 'req-71626c42-86bf-49b3-a091-b3c9d25d36ec'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.338441 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30614691f844edf8ff077f77141677b', 'request_id': 'req-71626c42-86bf-49b3-a091-b3c9d25d36ec'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.344440 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30614691f844edf8ff077f77141677b', 'request_id': 'req-71626c42-86bf-49b3-a091-b3c9d25d36ec'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.355222 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] The token cache is being invalidate because project b30614691f844edf8ff077f77141677b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:44.355506 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.356136 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.357649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1384/2818] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:44 2022] DELETE /identity/v3/projects/b30614691f844edf8ff077f77141677b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:44.362284 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9626c3-5ce6-4e58-8bed-eb7e3e534905 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:44.371799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9626c3-5ce6-4e58-8bed-eb7e3e534905 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:44.390418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:44.434556 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:44.435231 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:44.435516 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:44.435791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] PATH_INFO: `/v3/users/e809094c82ed45baa9b59adf0718f43a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:44.437397 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e809094c82ed45baa9b59adf0718f43a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:44.439372 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:44.486610 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e809094c82ed45baa9b59adf0718f43a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.494141 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e809094c82ed45baa9b59adf0718f43a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.501597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e809094c82ed45baa9b59adf0718f43a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.502366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e809094c82ed45baa9b59adf0718f43a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.514091 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-60fec93f-eb32-455e-ab68-d25c416c4d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e809094c82ed45baa9b59adf0718f43a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.519970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1435/2819] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:44 2022] DELETE /identity/v3/users/e809094c82ed45baa9b59adf0718f43a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:49:44.792112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:44.793593 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:44.807697 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626c42-86bf-49b3-a091-b3c9d25d36ec admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:44.870525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:44.871088 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:44.871373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:44.871651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] PATH_INFO: `/v3/projects/fdeccef2b33a43cc949adcccc9e68071` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:44.872795 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdeccef2b33a43cc949adcccc9e68071)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:44.874811 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:44.904713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:44.924558 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdeccef2b33a43cc949adcccc9e68071', 'request_id': 'req-5a24752d-b49b-4d23-96f7-59318fa3768a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.931969 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdeccef2b33a43cc949adcccc9e68071', 'request_id': 'req-5a24752d-b49b-4d23-96f7-59318fa3768a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.938142 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdeccef2b33a43cc949adcccc9e68071', 'request_id': 'req-5a24752d-b49b-4d23-96f7-59318fa3768a'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.949524 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] The token cache is being invalidate because project fdeccef2b33a43cc949adcccc9e68071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:44.949814 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.950455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a24752d-b49b-4d23-96f7-59318fa3768a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:44.951985 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1385/2820] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:44 2022] DELETE /identity/v3/projects/fdeccef2b33a43cc949adcccc9e68071 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:44.957396 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09ee88-255d-4a8d-b2d2-37f8d5c2257b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:44.957683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09ee88-255d-4a8d-b2d2-37f8d5c2257b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:44.958018 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09ee88-255d-4a8d-b2d2-37f8d5c2257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:44.977819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3d09ee88-255d-4a8d-b2d2-37f8d5c2257b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:44.979005 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3d09ee88-255d-4a8d-b2d2-37f8d5c2257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:45.038242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1436/2821] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:45.047493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624763ea-4ff4-493a-b8b9-844c00df576c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.047804 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624763ea-4ff4-493a-b8b9-844c00df576c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.048166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624763ea-4ff4-493a-b8b9-844c00df576c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.068257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-624763ea-4ff4-493a-b8b9-844c00df576c None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:45.069501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-624763ea-4ff4-493a-b8b9-844c00df576c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:45.077395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1386/2822] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:45.082535 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09ee88-255d-4a8d-b2d2-37f8d5c2257b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.092233 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09ee88-255d-4a8d-b2d2-37f8d5c2257b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.097887 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.098500 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.098781 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.099059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.099757 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:49:45.100065 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.101680 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.113321 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1437/2823] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:49:45.117577 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624763ea-4ff4-493a-b8b9-844c00df576c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.127543 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624763ea-4ff4-493a-b8b9-844c00df576c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.132061 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af3cc-e311-45aa-bc4d-59ccc261773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.132698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af3cc-e311-45aa-bc4d-59ccc261773e admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.132977 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af3cc-e311-45aa-bc4d-59ccc261773e admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.133397 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af3cc-e311-45aa-bc4d-59ccc261773e admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.134070 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870af3cc-e311-45aa-bc4d-59ccc261773e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.135946 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870af3cc-e311-45aa-bc4d-59ccc261773e admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.163154 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1387/2824] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:45 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:45.167342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.177520 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e902ee-2327-408d-b530-5684be031bc5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.205268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:45.240111 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731e891-10a1-4e8f-9bdd-a303c38105f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.240724 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731e891-10a1-4e8f-9bdd-a303c38105f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.241006 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731e891-10a1-4e8f-9bdd-a303c38105f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.241281 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731e891-10a1-4e8f-9bdd-a303c38105f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.241976 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731e891-10a1-4e8f-9bdd-a303c38105f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.243853 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731e891-10a1-4e8f-9bdd-a303c38105f6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.268708 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1438/2825] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:45 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:45.270922 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af3cc-e311-45aa-bc4d-59ccc261773e admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.281090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af3cc-e311-45aa-bc4d-59ccc261773e admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.286238 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081091f-9de6-4e7c-9c0a-a635401662b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.287111 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081091f-9de6-4e7c-9c0a-a635401662b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.287404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081091f-9de6-4e7c-9c0a-a635401662b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.287699 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081091f-9de6-4e7c-9c0a-a635401662b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.288392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081091f-9de6-4e7c-9c0a-a635401662b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.290026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081091f-9de6-4e7c-9c0a-a635401662b2 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.300671 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1388/2826] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:45.304739 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731e891-10a1-4e8f-9bdd-a303c38105f6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.316153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731e891-10a1-4e8f-9bdd-a303c38105f6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.320997 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe956981-23f0-43f3-8bab-9fe6cdaa99c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.321677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe956981-23f0-43f3-8bab-9fe6cdaa99c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.321985 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe956981-23f0-43f3-8bab-9fe6cdaa99c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.322255 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe956981-23f0-43f3-8bab-9fe6cdaa99c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.323031 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe956981-23f0-43f3-8bab-9fe6cdaa99c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.324661 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe956981-23f0-43f3-8bab-9fe6cdaa99c6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.334562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1439/2827] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:49:45.338677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081091f-9de6-4e7c-9c0a-a635401662b2 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.349128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081091f-9de6-4e7c-9c0a-a635401662b2 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.353857 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063235a8-8c1a-4b52-b3bb-fea5f8c9ac28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.354516 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063235a8-8c1a-4b52-b3bb-fea5f8c9ac28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.354803 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063235a8-8c1a-4b52-b3bb-fea5f8c9ac28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.355084 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063235a8-8c1a-4b52-b3bb-fea5f8c9ac28 admin admin] PATH_INFO: `/v3/projects/9d9fdd96ffeb4c8f9555e604363c5780/users/630c335a8799463dbd3f8409534a6ace/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.371253 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063235a8-8c1a-4b52-b3bb-fea5f8c9ac28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9fdd96ffeb4c8f9555e604363c5780, user_id=630c335a8799463dbd3f8409534a6ace, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.373921 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063235a8-8c1a-4b52-b3bb-fea5f8c9ac28 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.386673 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1389/2828] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:45 2022] PUT /identity/v3/projects/9d9fdd96ffeb4c8f9555e604363c5780/users/630c335a8799463dbd3f8409534a6ace/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:45.398092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbb895-ca0a-4d3e-b9f1-16c0739282a4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.398605 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbb895-ca0a-4d3e-b9f1-16c0739282a4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.398767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbb895-ca0a-4d3e-b9f1-16c0739282a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.428689 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c0cbb895-ca0a-4d3e-b9f1-16c0739282a4 None None] MFA Rules not processed for user `630c335a8799463dbd3f8409534a6ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:45.429948 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c0cbb895-ca0a-4d3e-b9f1-16c0739282a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:45.495940 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1440/2829] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:45.500367 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063235a8-8c1a-4b52-b3bb-fea5f8c9ac28 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.560965 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063235a8-8c1a-4b52-b3bb-fea5f8c9ac28 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.566028 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736389bd-790b-48a2-8781-f23038968ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.566636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736389bd-790b-48a2-8781-f23038968ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.566914 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736389bd-790b-48a2-8781-f23038968ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.567183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736389bd-790b-48a2-8781-f23038968ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.567877 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736389bd-790b-48a2-8781-f23038968ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.570288 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736389bd-790b-48a2-8781-f23038968ee1 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.598110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1390/2830] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:45 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:45.602134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbb895-ca0a-4d3e-b9f1-16c0739282a4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.612569 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbb895-ca0a-4d3e-b9f1-16c0739282a4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.667288 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5df5c6-3855-413f-8b26-249e3f550540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.667819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5df5c6-3855-413f-8b26-249e3f550540 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.668095 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5df5c6-3855-413f-8b26-249e3f550540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.668363 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5df5c6-3855-413f-8b26-249e3f550540 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.669019 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5df5c6-3855-413f-8b26-249e3f550540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.670923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5df5c6-3855-413f-8b26-249e3f550540 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.691913 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1441/2831] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:45 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:45.696092 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736389bd-790b-48a2-8781-f23038968ee1 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.706991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736389bd-790b-48a2-8781-f23038968ee1 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.713188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4a1b-10b6-4c87-a613-c1f52a58abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.713892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4a1b-10b6-4c87-a613-c1f52a58abbd admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.714188 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4a1b-10b6-4c87-a613-c1f52a58abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.714493 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4a1b-10b6-4c87-a613-c1f52a58abbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.715282 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b4a1b-10b6-4c87-a613-c1f52a58abbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.716912 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b4a1b-10b6-4c87-a613-c1f52a58abbd admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.728399 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1391/2832] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:45.732687 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5df5c6-3855-413f-8b26-249e3f550540 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.742811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5df5c6-3855-413f-8b26-249e3f550540 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.747371 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd8f14-843f-4610-a6c9-d03e90fb2973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.748015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd8f14-843f-4610-a6c9-d03e90fb2973 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.748292 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd8f14-843f-4610-a6c9-d03e90fb2973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.748563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd8f14-843f-4610-a6c9-d03e90fb2973 admin admin] PATH_INFO: `/v3/projects/8b96da7e56994534ae00dc504ddbdb4f/users/26ea90c24e8249ff9f5c8f2aa966de8c/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.763134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd8f14-843f-4610-a6c9-d03e90fb2973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b96da7e56994534ae00dc504ddbdb4f, user_id=26ea90c24e8249ff9f5c8f2aa966de8c, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.765778 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd8f14-843f-4610-a6c9-d03e90fb2973 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.779110 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1442/2833] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:45 2022] PUT /identity/v3/projects/8b96da7e56994534ae00dc504ddbdb4f/users/26ea90c24e8249ff9f5c8f2aa966de8c/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:45.783903 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4a1b-10b6-4c87-a613-c1f52a58abbd admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.794345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4a1b-10b6-4c87-a613-c1f52a58abbd admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.815806 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:45.854798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2491e-05e3-4115-8216-74e74b10b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.855420 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2491e-05e3-4115-8216-74e74b10b795 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.855706 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2491e-05e3-4115-8216-74e74b10b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.855986 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2491e-05e3-4115-8216-74e74b10b795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.856677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2491e-05e3-4115-8216-74e74b10b795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.858305 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2491e-05e3-4115-8216-74e74b10b795 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.868059 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1392/2834] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:45.872214 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd8f14-843f-4610-a6c9-d03e90fb2973 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:45.878817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:45.885017 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd8f14-843f-4610-a6c9-d03e90fb2973 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:45.889955 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:45.890749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.891618 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.892007 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] PATH_INFO: `/v3/projects/8b96da7e56994534ae00dc504ddbdb4f/users/26ea90c24e8249ff9f5c8f2aa966de8c/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.893981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b96da7e56994534ae00dc504ddbdb4f, user_id=26ea90c24e8249ff9f5c8f2aa966de8c, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:45.896589 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:45.906651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1443/2835] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:45 2022] PUT /identity/v3/projects/8b96da7e56994534ae00dc504ddbdb4f/users/26ea90c24e8249ff9f5c8f2aa966de8c/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:45.916799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732935b8-9915-4466-b921-8ffb35d1204b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:45.917112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732935b8-9915-4466-b921-8ffb35d1204b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:45.917520 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732935b8-9915-4466-b921-8ffb35d1204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:45.945272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-732935b8-9915-4466-b921-8ffb35d1204b None None] MFA Rules not processed for user `26ea90c24e8249ff9f5c8f2aa966de8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:45.946514 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-732935b8-9915-4466-b921-8ffb35d1204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:46.013931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1393/2836] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:46.025080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:46.026792 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:46.082614 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c692e-8907-40ae-96c4-482bcbf7ae34 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:46.086179 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fbb8c-a133-433a-b160-c9b8118f2482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:46.086813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fbb8c-a133-433a-b160-c9b8118f2482 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:46.087092 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fbb8c-a133-433a-b160-c9b8118f2482 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:46.087381 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fbb8c-a133-433a-b160-c9b8118f2482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:46.096136 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fbb8c-a133-433a-b160-c9b8118f2482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:46.097563 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fbb8c-a133-433a-b160-c9b8118f2482 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:46.106707 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1444/2837] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:47.442307 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732935b8-9915-4466-b921-8ffb35d1204b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:47.504764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732935b8-9915-4466-b921-8ffb35d1204b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:47.509183 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:47.509850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:47.510134 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:47.510411 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] PATH_INFO: `/v3/users/630c335a8799463dbd3f8409534a6ace` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:47.511582 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=630c335a8799463dbd3f8409534a6ace)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:47.513525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:47.559677 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '630c335a8799463dbd3f8409534a6ace'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:47.561276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:47.567713 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '630c335a8799463dbd3f8409534a6ace'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:47.568563 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '630c335a8799463dbd3f8409534a6ace'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:47.580792 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '630c335a8799463dbd3f8409534a6ace'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:47.581458 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '630c335a8799463dbd3f8409534a6ace'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:47.602739 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1394/2838] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:47 2022] DELETE /identity/v3/users/630c335a8799463dbd3f8409534a6ace => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:47.607319 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019926-e5d7-44a6-bd63-016a8b4814aa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:47.607826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019926-e5d7-44a6-bd63-016a8b4814aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:47.607934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019926-e5d7-44a6-bd63-016a8b4814aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:47.627968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b5019926-e5d7-44a6-bd63-016a8b4814aa None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:47.629156 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b5019926-e5d7-44a6-bd63-016a8b4814aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:47.687856 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1445/2839] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:47.698221 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:47.699643 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:47.756580 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db552c22-c0a0-4d55-8601-7e1842efce66 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:47.760208 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce11f9f-5cc6-48a5-82d4-a05a35628f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:47.760851 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11f9f-5cc6-48a5-82d4-a05a35628f91 service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:47.761143 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11f9f-5cc6-48a5-82d4-a05a35628f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:47.761453 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce11f9f-5cc6-48a5-82d4-a05a35628f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:47.771057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce11f9f-5cc6-48a5-82d4-a05a35628f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:47.772485 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce11f9f-5cc6-48a5-82d4-a05a35628f91 service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:47.782714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1395/2840] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:48.046655 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5019926-e5d7-44a6-bd63-016a8b4814aa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:48.048248 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b5019926-e5d7-44a6-bd63-016a8b4814aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:48.060743 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5019926-e5d7-44a6-bd63-016a8b4814aa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:48.066574 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:48.067229 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:48.067559 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:48.067844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] PATH_INFO: `/v3/projects/9d9fdd96ffeb4c8f9555e604363c5780` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:48.069135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d9fdd96ffeb4c8f9555e604363c5780)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:48.071070 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:48.119230 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d9fdd96ffeb4c8f9555e604363c5780', 'request_id': 'req-9d99b845-5e30-429a-8ac4-f21409b23613'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.124521 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d9fdd96ffeb4c8f9555e604363c5780', 'request_id': 'req-9d99b845-5e30-429a-8ac4-f21409b23613'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.131934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d9fdd96ffeb4c8f9555e604363c5780', 'request_id': 'req-9d99b845-5e30-429a-8ac4-f21409b23613'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.143346 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] The token cache is being invalidate because project 9d9fdd96ffeb4c8f9555e604363c5780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:48.143632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.144257 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.145768 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1446/2841] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:48 2022] DELETE /identity/v3/projects/9d9fdd96ffeb4c8f9555e604363c5780 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:48.149742 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce11f9f-5cc6-48a5-82d4-a05a35628f91 service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:48.159923 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce11f9f-5cc6-48a5-82d4-a05a35628f91 service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:48.217032 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:48.217710 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:48.217998 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:48.218276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] PATH_INFO: `/v3/users/26ea90c24e8249ff9f5c8f2aa966de8c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:48.219595 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ea90c24e8249ff9f5c8f2aa966de8c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:48.221564 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:48.265632 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ea90c24e8249ff9f5c8f2aa966de8c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.273727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ea90c24e8249ff9f5c8f2aa966de8c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.274345 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ea90c24e8249ff9f5c8f2aa966de8c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.285395 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:48.286601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ea90c24e8249ff9f5c8f2aa966de8c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.287298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-80ece3ce-b08e-4eb2-9c77-e9cbf73b4877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ea90c24e8249ff9f5c8f2aa966de8c'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.301265 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1396/2842] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:48 2022] DELETE /identity/v3/users/26ea90c24e8249ff9f5c8f2aa966de8c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:48.611892 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:48.613388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:48.629493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b845-5e30-429a-8ac4-f21409b23613 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:48.642409 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:48.711982 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:48.712651 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:48.712931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:48.713210 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] PATH_INFO: `/v3/projects/8b96da7e56994534ae00dc504ddbdb4f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:48.714390 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b96da7e56994534ae00dc504ddbdb4f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:48.716334 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:48.770980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b96da7e56994534ae00dc504ddbdb4f', 'request_id': 'req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.776367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b96da7e56994534ae00dc504ddbdb4f', 'request_id': 'req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.784539 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b96da7e56994534ae00dc504ddbdb4f', 'request_id': 'req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.795907 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] The token cache is being invalidate because project 8b96da7e56994534ae00dc504ddbdb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:48.796194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.796818 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0aa89a33-7c46-4c3a-951b-2a08afd0ecbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:48.798392 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1447/2843] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:48 2022] DELETE /identity/v3/projects/8b96da7e56994534ae00dc504ddbdb4f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:49:48.803513 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42d42a-c95e-45d0-8829-75457812bc2b None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:48.804019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42d42a-c95e-45d0-8829-75457812bc2b None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:48.804133 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42d42a-c95e-45d0-8829-75457812bc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:48.825378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-6b42d42a-c95e-45d0-8829-75457812bc2b None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:48.826508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6b42d42a-c95e-45d0-8829-75457812bc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:48.886219 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1397/2844] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:48.894746 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba926b-d78d-4249-873a-c99f892fa11a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:48.895034 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba926b-d78d-4249-873a-c99f892fa11a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:48.895338 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba926b-d78d-4249-873a-c99f892fa11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:48.916077 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f5ba926b-d78d-4249-873a-c99f892fa11a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:48.917660 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f5ba926b-d78d-4249-873a-c99f892fa11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:48.926193 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1448/2845] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:48.930462 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42d42a-c95e-45d0-8829-75457812bc2b None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:48.941038 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42d42a-c95e-45d0-8829-75457812bc2b None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:48.945652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:48.946257 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:48.946539 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:48.946810 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] PATH_INFO: `/v3/domains` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:48.947497 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80276) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:49:48.947799 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:48.949406 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:48.961374 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1398/2846] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:49:48.965522 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba926b-d78d-4249-873a-c99f892fa11a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:48.975089 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba926b-d78d-4249-873a-c99f892fa11a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:48.979682 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851981a-f7a6-4972-b8ee-709ec993da35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:48.980283 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851981a-f7a6-4972-b8ee-709ec993da35 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:48.980565 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851981a-f7a6-4972-b8ee-709ec993da35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:48.980839 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851981a-f7a6-4972-b8ee-709ec993da35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:48.981534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851981a-f7a6-4972-b8ee-709ec993da35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:48.983401 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851981a-f7a6-4972-b8ee-709ec993da35 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.010643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1449/2847] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:48 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:49:49.014899 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.026267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a759-4822-4c1a-868f-fe4654580045 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.083860 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53f5fb-1e2e-4434-b89e-36a054e8f89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.084389 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f5fb-1e2e-4434-b89e-36a054e8f89a admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.084669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f5fb-1e2e-4434-b89e-36a054e8f89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.084941 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f5fb-1e2e-4434-b89e-36a054e8f89a admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.085636 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53f5fb-1e2e-4434-b89e-36a054e8f89a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.087533 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53f5fb-1e2e-4434-b89e-36a054e8f89a admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.107917 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1399/2848] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:49 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:49:49.112172 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851981a-f7a6-4972-b8ee-709ec993da35 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.122461 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851981a-f7a6-4972-b8ee-709ec993da35 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.128085 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c782f-424f-4116-9334-b599727be7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.128690 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c782f-424f-4116-9334-b599727be7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.128970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c782f-424f-4116-9334-b599727be7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.129242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c782f-424f-4116-9334-b599727be7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.129970 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c782f-424f-4116-9334-b599727be7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.131921 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c782f-424f-4116-9334-b599727be7fc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.142811 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1450/2849] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:49.146805 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53f5fb-1e2e-4434-b89e-36a054e8f89a admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.156727 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53f5fb-1e2e-4434-b89e-36a054e8f89a admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.161386 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9ff5b-7f01-4371-8fa1-555ad4ed4fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.161990 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ff5b-7f01-4371-8fa1-555ad4ed4fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.162275 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ff5b-7f01-4371-8fa1-555ad4ed4fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.162550 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ff5b-7f01-4371-8fa1-555ad4ed4fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.163242 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9ff5b-7f01-4371-8fa1-555ad4ed4fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.164824 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9ff5b-7f01-4371-8fa1-555ad4ed4fb9 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.175247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1400/2850] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:49.179224 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c782f-424f-4116-9334-b599727be7fc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.190135 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c782f-424f-4116-9334-b599727be7fc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.195320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f090a7-acc4-4fcd-8f17-ec158fb090ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.195968 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f090a7-acc4-4fcd-8f17-ec158fb090ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.196252 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f090a7-acc4-4fcd-8f17-ec158fb090ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.196530 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f090a7-acc4-4fcd-8f17-ec158fb090ca admin admin] PATH_INFO: `/v3/projects/d1f2182aadf54e4c98f8d9d24745c359/users/468020fcb93e4e16a79d01b53762de56/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.212192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f090a7-acc4-4fcd-8f17-ec158fb090ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f2182aadf54e4c98f8d9d24745c359, user_id=468020fcb93e4e16a79d01b53762de56, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.214865 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f090a7-acc4-4fcd-8f17-ec158fb090ca admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.227200 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1451/2851] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:49 2022] PUT /identity/v3/projects/d1f2182aadf54e4c98f8d9d24745c359/users/468020fcb93e4e16a79d01b53762de56/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:49.237984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84e8c3-d245-424a-a76a-075c7a16fe9c None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.238276 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84e8c3-d245-424a-a76a-075c7a16fe9c None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.238552 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84e8c3-d245-424a-a76a-075c7a16fe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.267223 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-7f84e8c3-d245-424a-a76a-075c7a16fe9c None None] MFA Rules not processed for user `468020fcb93e4e16a79d01b53762de56`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:49.268436 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-7f84e8c3-d245-424a-a76a-075c7a16fe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:49.309464 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:49.333651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1401/2852] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:49.338192 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f090a7-acc4-4fcd-8f17-ec158fb090ca admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.383054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:49.401993 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f090a7-acc4-4fcd-8f17-ec158fb090ca admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.407217 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c8b0d-55f3-4b80-ae7a-36319af524f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.407817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c8b0d-55f3-4b80-ae7a-36319af524f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.408098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c8b0d-55f3-4b80-ae7a-36319af524f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.408366 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c8b0d-55f3-4b80-ae7a-36319af524f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.409016 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c8b0d-55f3-4b80-ae7a-36319af524f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.410920 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c8b0d-55f3-4b80-ae7a-36319af524f8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.436176 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1452/2853] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:49 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:49.440278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84e8c3-d245-424a-a76a-075c7a16fe9c None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.451404 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84e8c3-d245-424a-a76a-075c7a16fe9c None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.507984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3502e10-3fff-47e0-931d-739b7eb8baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.508612 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3502e10-3fff-47e0-931d-739b7eb8baa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.508892 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3502e10-3fff-47e0-931d-739b7eb8baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.509164 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3502e10-3fff-47e0-931d-739b7eb8baa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.509855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3502e10-3fff-47e0-931d-739b7eb8baa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.511764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3502e10-3fff-47e0-931d-739b7eb8baa6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.534371 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1402/2854] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:49 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:49:49.538434 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c8b0d-55f3-4b80-ae7a-36319af524f8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.548493 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c8b0d-55f3-4b80-ae7a-36319af524f8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.552999 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29191ed5-683a-4c04-a286-527107846195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.553632 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191ed5-683a-4c04-a286-527107846195 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.553912 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191ed5-683a-4c04-a286-527107846195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.554181 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191ed5-683a-4c04-a286-527107846195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.554864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29191ed5-683a-4c04-a286-527107846195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.556436 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29191ed5-683a-4c04-a286-527107846195 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.567186 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1453/2855] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 06:49:49.571166 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3502e10-3fff-47e0-931d-739b7eb8baa6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.581956 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3502e10-3fff-47e0-931d-739b7eb8baa6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.586626 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76c168-19a1-4420-82bf-feb3d06109e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.587278 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76c168-19a1-4420-82bf-feb3d06109e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.587557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76c168-19a1-4420-82bf-feb3d06109e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.587830 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76c168-19a1-4420-82bf-feb3d06109e6 admin admin] PATH_INFO: `/v3/projects/fd379792b3ed41bda7ec90cb2f296f8b/users/bf775f6c918047aa8056399ee628c646/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.603575 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76c168-19a1-4420-82bf-feb3d06109e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd379792b3ed41bda7ec90cb2f296f8b, user_id=bf775f6c918047aa8056399ee628c646, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.606554 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76c168-19a1-4420-82bf-feb3d06109e6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.621123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1403/2856] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:49 2022] PUT /identity/v3/projects/fd379792b3ed41bda7ec90cb2f296f8b/users/bf775f6c918047aa8056399ee628c646/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:49.625022 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29191ed5-683a-4c04-a286-527107846195 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.635835 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29191ed5-683a-4c04-a286-527107846195 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.691059 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293147a-04fe-4482-be9b-b493dc342a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.691675 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293147a-04fe-4482-be9b-b493dc342a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.691962 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293147a-04fe-4482-be9b-b493dc342a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.692239 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293147a-04fe-4482-be9b-b493dc342a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.692934 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293147a-04fe-4482-be9b-b493dc342a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.694591 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293147a-04fe-4482-be9b-b493dc342a2b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.704796 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1454/2857] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:49.708953 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76c168-19a1-4420-82bf-feb3d06109e6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.719676 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76c168-19a1-4420-82bf-feb3d06109e6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.724484 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.725157 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.725477 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.725755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] PATH_INFO: `/v3/projects/fd379792b3ed41bda7ec90cb2f296f8b/users/bf775f6c918047aa8056399ee628c646/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.727973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd379792b3ed41bda7ec90cb2f296f8b, user_id=bf775f6c918047aa8056399ee628c646, role_id=ae9c1d7a6a26488f8c6a15ecfe70fa1c)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.730716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.741896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1404/2858] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:49 2022] PUT /identity/v3/projects/fd379792b3ed41bda7ec90cb2f296f8b/users/bf775f6c918047aa8056399ee628c646/roles/ae9c1d7a6a26488f8c6a15ecfe70fa1c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:49.753980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49da5c-d01f-4529-a084-02fe63066f1d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.754268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49da5c-d01f-4529-a084-02fe63066f1d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.754548 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49da5c-d01f-4529-a084-02fe63066f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.782996 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-de49da5c-d01f-4529-a084-02fe63066f1d None None] MFA Rules not processed for user `bf775f6c918047aa8056399ee628c646`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:49.784244 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-de49da5c-d01f-4529-a084-02fe63066f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:49.852320 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1455/2859] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:49.863546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:49.865381 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:49.921725 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb545a-be28-44ae-9645-65a2308192a3 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:49.926025 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5fc07-1d48-4fda-8112-60812fa84a53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:49.926637 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5fc07-1d48-4fda-8112-60812fa84a53 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:49.926920 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5fc07-1d48-4fda-8112-60812fa84a53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:49.927202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5fc07-1d48-4fda-8112-60812fa84a53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:49.936442 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5fc07-1d48-4fda-8112-60812fa84a53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:49.937982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5fc07-1d48-4fda-8112-60812fa84a53 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:49.947652 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1405/2860] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:50.491927 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49da5c-d01f-4529-a084-02fe63066f1d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:50.533285 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:50.555014 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49da5c-d01f-4529-a084-02fe63066f1d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:50.559534 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:50.560168 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:50.560450 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:50.560723 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] PATH_INFO: `/v3/users/468020fcb93e4e16a79d01b53762de56` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:50.561893 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=468020fcb93e4e16a79d01b53762de56)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:50.564750 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:50.609995 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468020fcb93e4e16a79d01b53762de56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:50.616923 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468020fcb93e4e16a79d01b53762de56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:50.624403 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468020fcb93e4e16a79d01b53762de56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:50.625021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468020fcb93e4e16a79d01b53762de56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:50.637634 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468020fcb93e4e16a79d01b53762de56'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:50.644662 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1456/2861] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:50 2022] DELETE /identity/v3/users/468020fcb93e4e16a79d01b53762de56 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:50.649090 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12400256-7767-4a9f-8cda-e850955cdb43 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:50.649403 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12400256-7767-4a9f-8cda-e850955cdb43 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:50.649687 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12400256-7767-4a9f-8cda-e850955cdb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:50.668372 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-12400256-7767-4a9f-8cda-e850955cdb43 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:50.669601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-12400256-7767-4a9f-8cda-e850955cdb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:50.701649 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:50.729938 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1406/2862] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:50.740441 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:50.741880 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:50.799573 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88bc42-0e54-42be-b861-061c1a686aa7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:50.803226 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0fb90-72d5-4d19-8559-2abff8ad5765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:50.803838 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0fb90-72d5-4d19-8559-2abff8ad5765 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:50.804123 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0fb90-72d5-4d19-8559-2abff8ad5765 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:50.804400 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0fb90-72d5-4d19-8559-2abff8ad5765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:50.813379 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0fb90-72d5-4d19-8559-2abff8ad5765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:50.814807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0fb90-72d5-4d19-8559-2abff8ad5765 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:50.824517 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1457/2863] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:49:51.088510 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12400256-7767-4a9f-8cda-e850955cdb43 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:51.089942 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-12400256-7767-4a9f-8cda-e850955cdb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:51.103224 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12400256-7767-4a9f-8cda-e850955cdb43 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:51.107894 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:51.108540 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:51.108822 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:51.109097 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] PATH_INFO: `/v3/projects/d1f2182aadf54e4c98f8d9d24745c359` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:51.111361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f2182aadf54e4c98f8d9d24745c359)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:51.113272 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:51.161424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f2182aadf54e4c98f8d9d24745c359', 'request_id': 'req-fd079996-4654-46c4-bb9a-e32fadf4a986'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.168716 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f2182aadf54e4c98f8d9d24745c359', 'request_id': 'req-fd079996-4654-46c4-bb9a-e32fadf4a986'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.174991 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f2182aadf54e4c98f8d9d24745c359', 'request_id': 'req-fd079996-4654-46c4-bb9a-e32fadf4a986'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.185887 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] The token cache is being invalidate because project d1f2182aadf54e4c98f8d9d24745c359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:51.186174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.186798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.188630 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1407/2864] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:51 2022] DELETE /identity/v3/projects/d1f2182aadf54e4c98f8d9d24745c359 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:51.192813 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0fb90-72d5-4d19-8559-2abff8ad5765 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:51.202791 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0fb90-72d5-4d19-8559-2abff8ad5765 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:51.258673 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:51.259314 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:51.259598 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:51.259876 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] PATH_INFO: `/v3/users/bf775f6c918047aa8056399ee628c646` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:51.261037 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf775f6c918047aa8056399ee628c646)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:51.262986 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:51.264428 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:51.312849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf775f6c918047aa8056399ee628c646'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.319947 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf775f6c918047aa8056399ee628c646'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.327054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf775f6c918047aa8056399ee628c646'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.327674 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf775f6c918047aa8056399ee628c646'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.339227 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88ff4c2f-1fea-495c-b9a9-f623c8204ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf775f6c918047aa8056399ee628c646'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.345478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1458/2865] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:49:51 2022] DELETE /identity/v3/users/bf775f6c918047aa8056399ee628c646 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:51.632954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:51.634392 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:51.649049 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd079996-4654-46c4-bb9a-e32fadf4a986 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:51.716562 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:51.717548 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:51.717975 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:51.718391 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] PATH_INFO: `/v3/projects/fd379792b3ed41bda7ec90cb2f296f8b` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:51.720784 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd379792b3ed41bda7ec90cb2f296f8b)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:51.723866 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:51.779152 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd379792b3ed41bda7ec90cb2f296f8b', 'request_id': 'req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.787128 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd379792b3ed41bda7ec90cb2f296f8b', 'request_id': 'req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.791825 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:51.793051 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd379792b3ed41bda7ec90cb2f296f8b', 'request_id': 'req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.806095 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] The token cache is being invalidate because project fd379792b3ed41bda7ec90cb2f296f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:49:51.806387 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.807021 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-7c9ec84e-66c8-47ff-9f45-89c2f1637e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:49:51.808521 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1408/2866] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:49:51 2022] DELETE /identity/v3/projects/fd379792b3ed41bda7ec90cb2f296f8b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:51.813668 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142283e-ba6d-4a3a-9083-c89423495c9a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:51.814166 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142283e-ba6d-4a3a-9083-c89423495c9a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:51.814418 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142283e-ba6d-4a3a-9083-c89423495c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:51.835133 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a142283e-ba6d-4a3a-9083-c89423495c9a None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:51.836348 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a142283e-ba6d-4a3a-9083-c89423495c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:51.896356 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1459/2867] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:51.904169 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4204-a224-4a6a-920a-9862e73a0729 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:51.904460 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4204-a224-4a6a-920a-9862e73a0729 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:51.904755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd4204-a224-4a6a-920a-9862e73a0729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:51.924318 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-eccd4204-a224-4a6a-920a-9862e73a0729 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:51.925571 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-eccd4204-a224-4a6a-920a-9862e73a0729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:51.933529 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1409/2868] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:51.938079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142283e-ba6d-4a3a-9083-c89423495c9a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:51.948510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142283e-ba6d-4a3a-9083-c89423495c9a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:51.953040 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:51.954773 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:51.955056 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:51.955362 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:51.956082 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:49:51.956388 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:51.958003 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:51.969864 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1460/2869] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:49:51.974141 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd4204-a224-4a6a-920a-9862e73a0729 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:51.983302 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd4204-a224-4a6a-920a-9862e73a0729 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:51.988178 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8e057-b965-4400-adaf-13ce06603d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:51.988781 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8e057-b965-4400-adaf-13ce06603d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:51.989063 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8e057-b965-4400-adaf-13ce06603d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:51.989368 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8e057-b965-4400-adaf-13ce06603d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:51.990030 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8e057-b965-4400-adaf-13ce06603d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:51.991915 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8e057-b965-4400-adaf-13ce06603d29 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:52.021718 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1410/2870] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:49:51 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:52.025799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:52.036395 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b11e10-e6d0-4e28-96c4-122f63bab1aa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:52.093980 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6b3ea-eee1-466b-9579-725071020f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:52.094760 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b3ea-eee1-466b-9579-725071020f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:52.095048 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b3ea-eee1-466b-9579-725071020f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:52.095325 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b3ea-eee1-466b-9579-725071020f8f admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:52.096021 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6b3ea-eee1-466b-9579-725071020f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:52.097954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6b3ea-eee1-466b-9579-725071020f8f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:52.109029 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:49:52.121431 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1461/2871] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:49:52 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:49:52.126112 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8e057-b965-4400-adaf-13ce06603d29 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:52.136414 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8e057-b965-4400-adaf-13ce06603d29 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:52.141896 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90c0e2-d1b9-4ed6-b796-85b60859a9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:52.142503 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90c0e2-d1b9-4ed6-b796-85b60859a9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:52.142783 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90c0e2-d1b9-4ed6-b796-85b60859a9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:52.143057 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90c0e2-d1b9-4ed6-b796-85b60859a9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:52.143764 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90c0e2-d1b9-4ed6-b796-85b60859a9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:52.145421 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90c0e2-d1b9-4ed6-b796-85b60859a9f8 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:52.155108 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1411/2872] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:52.159287 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6b3ea-eee1-466b-9579-725071020f8f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:52.169197 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6b3ea-eee1-466b-9579-725071020f8f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:52.174423 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721435b-f422-4917-8a43-763c76cba02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:52.175054 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721435b-f422-4917-8a43-763c76cba02e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:52.175335 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721435b-f422-4917-8a43-763c76cba02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:52.175608 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721435b-f422-4917-8a43-763c76cba02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:52.176293 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721435b-f422-4917-8a43-763c76cba02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:52.177898 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721435b-f422-4917-8a43-763c76cba02e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:52.189784 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1462/2873] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:49:52.194042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90c0e2-d1b9-4ed6-b796-85b60859a9f8 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:52.205071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90c0e2-d1b9-4ed6-b796-85b60859a9f8 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:52.210122 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:52.210777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:52.211060 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:52.211337 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] PATH_INFO: `/v3/projects/410291b1a080456e945f3c4bcc59f082/users/801a30fca4b64a6598dbaf17ddc5121b/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:52.225995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=410291b1a080456e945f3c4bcc59f082, user_id=801a30fca4b64a6598dbaf17ddc5121b, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:52.228617 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:52.241835 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1412/2874] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:49:52 2022] PUT /identity/v3/projects/410291b1a080456e945f3c4bcc59f082/users/801a30fca4b64a6598dbaf17ddc5121b/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:49:52.252981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a56442-be41-4eb9-a84c-2cc0bf2d7a4d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:52.253272 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a56442-be41-4eb9-a84c-2cc0bf2d7a4d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:52.253584 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a56442-be41-4eb9-a84c-2cc0bf2d7a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:52.282046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-06a56442-be41-4eb9-a84c-2cc0bf2d7a4d None None] MFA Rules not processed for user `801a30fca4b64a6598dbaf17ddc5121b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:49:52.283698 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-06a56442-be41-4eb9-a84c-2cc0bf2d7a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:52.348957 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1463/2875] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:49:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:49:52.360439 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:49:52.362139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:49:52.420624 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b830e39-2aea-480b-8061-288d07e48f0c admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:49:52.424749 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71bc99-1301-4b9e-b71d-d789475cfd52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:49:52.425398 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71bc99-1301-4b9e-b71d-d789475cfd52 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:49:52.425683 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71bc99-1301-4b9e-b71d-d789475cfd52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:49:52.425959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71bc99-1301-4b9e-b71d-d789475cfd52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:49:52.434330 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71bc99-1301-4b9e-b71d-d789475cfd52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:49:52.435755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71bc99-1301-4b9e-b71d-d789475cfd52 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:49:52.445546 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1413/2876] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:49:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:50:02.346189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=39 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:02.441611 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:11.065870 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a56442-be41-4eb9-a84c-2cc0bf2d7a4d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:11.128316 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a56442-be41-4eb9-a84c-2cc0bf2d7a4d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:11.133823 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:11.134460 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:11.134742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:11.135015 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] PATH_INFO: `/v3/users/801a30fca4b64a6598dbaf17ddc5121b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:11.136173 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] RBAC: Authorizing `identity:delete_user(user_id=801a30fca4b64a6598dbaf17ddc5121b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:11.138109 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:11.183507 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801a30fca4b64a6598dbaf17ddc5121b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.190189 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801a30fca4b64a6598dbaf17ddc5121b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.198098 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801a30fca4b64a6598dbaf17ddc5121b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.198712 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801a30fca4b64a6598dbaf17ddc5121b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.210360 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801a30fca4b64a6598dbaf17ddc5121b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.216177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1464/2877] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:50:11 2022] DELETE /identity/v3/users/801a30fca4b64a6598dbaf17ddc5121b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:50:11.220853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c8fb3-0006-4e11-9eb7-c794dc4372be None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:11.221384 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c8fb3-0006-4e11-9eb7-c794dc4372be None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:11.221508 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c8fb3-0006-4e11-9eb7-c794dc4372be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:11.242160 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-127c8fb3-0006-4e11-9eb7-c794dc4372be None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:11.243355 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-127c8fb3-0006-4e11-9eb7-c794dc4372be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:11.301285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1414/2878] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:11.312742 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:11.314256 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:11.372039 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b4a09-2d15-4a1c-a14c-9c5a286dbeca admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:11.375643 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a8274-e2ec-4732-959f-dc7eba37c92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:11.376246 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a8274-e2ec-4732-959f-dc7eba37c92c service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:11.376529 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a8274-e2ec-4732-959f-dc7eba37c92c service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:11.376807 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a8274-e2ec-4732-959f-dc7eba37c92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:11.386367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a8274-e2ec-4732-959f-dc7eba37c92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:11.394570 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a8274-e2ec-4732-959f-dc7eba37c92c service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:11.404603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1465/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:50:11.684756 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c8fb3-0006-4e11-9eb7-c794dc4372be None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:11.686180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-127c8fb3-0006-4e11-9eb7-c794dc4372be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:11.698745 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c8fb3-0006-4e11-9eb7-c794dc4372be None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:11.703525 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:11.704179 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:11.704481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:11.704863 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] PATH_INFO: `/v3/projects/410291b1a080456e945f3c4bcc59f082` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:11.705984 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=410291b1a080456e945f3c4bcc59f082)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:11.707924 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:11.755447 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410291b1a080456e945f3c4bcc59f082', 'request_id': 'req-5a2123aa-0cec-43f6-9d36-34161d5c3014'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.763602 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410291b1a080456e945f3c4bcc59f082', 'request_id': 'req-5a2123aa-0cec-43f6-9d36-34161d5c3014'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.769365 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410291b1a080456e945f3c4bcc59f082', 'request_id': 'req-5a2123aa-0cec-43f6-9d36-34161d5c3014'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.781168 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] The token cache is being invalidate because project 410291b1a080456e945f3c4bcc59f082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80276) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:50:11.781482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.782106 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-5a2123aa-0cec-43f6-9d36-34161d5c3014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:11.783587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1415/2880] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:50:11 2022] DELETE /identity/v3/projects/410291b1a080456e945f3c4bcc59f082 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:50:11.789232 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6272e-dc73-4a5c-9019-94dc495d9927 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:11.789562 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6272e-dc73-4a5c-9019-94dc495d9927 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:11.789843 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6272e-dc73-4a5c-9019-94dc495d9927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:11.809278 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b8a6272e-dc73-4a5c-9019-94dc495d9927 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:11.810506 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b8a6272e-dc73-4a5c-9019-94dc495d9927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:11.828549 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:11.877879 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1466/2881] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:11.885951 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76c821-3a61-4b58-a41c-eb87f70356a0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:11.886327 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76c821-3a61-4b58-a41c-eb87f70356a0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:11.886651 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76c821-3a61-4b58-a41c-eb87f70356a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:11.907009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-4c76c821-3a61-4b58-a41c-eb87f70356a0 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:11.908215 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-4c76c821-3a61-4b58-a41c-eb87f70356a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:11.916798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1416/2882] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:11.921114 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6272e-dc73-4a5c-9019-94dc495d9927 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:11.930966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6272e-dc73-4a5c-9019-94dc495d9927 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:11.935597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:11.936204 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:11.936489 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:11.936767 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:11.937497 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 06:50:11.937809 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:11.939394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:11.951073 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1467/2883] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Jul 28 06:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 06:50:11.955267 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76c821-3a61-4b58-a41c-eb87f70356a0 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:11.965214 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76c821-3a61-4b58-a41c-eb87f70356a0 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:11.969973 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da60a7-b7c3-4e57-874e-16138cfd5d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:11.970581 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da60a7-b7c3-4e57-874e-16138cfd5d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:11.970868 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da60a7-b7c3-4e57-874e-16138cfd5d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:11.971146 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da60a7-b7c3-4e57-874e-16138cfd5d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:11.971818 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da60a7-b7c3-4e57-874e-16138cfd5d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:11.973760 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da60a7-b7c3-4e57-874e-16138cfd5d57 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:12.000723 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1417/2884] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Jul 28 06:50:11 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 06:50:12.005041 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:12.015518 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80710d9b-1a48-4d30-b4b1-bc989c53e179 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:12.073240 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d895ff-58e6-4ecd-adcc-f4dc887fa533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:12.073935 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d895ff-58e6-4ecd-adcc-f4dc887fa533 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:12.074153 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d895ff-58e6-4ecd-adcc-f4dc887fa533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:12.074424 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d895ff-58e6-4ecd-adcc-f4dc887fa533 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:12.075118 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d895ff-58e6-4ecd-adcc-f4dc887fa533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:12.077047 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d895ff-58e6-4ecd-adcc-f4dc887fa533 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:12.098318 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1468/2885] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Jul 28 06:50:12 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 06:50:12.102698 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da60a7-b7c3-4e57-874e-16138cfd5d57 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:12.112970 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da60a7-b7c3-4e57-874e-16138cfd5d57 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:12.118041 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b64861-8fae-4a3f-a18f-f19546e33da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:12.118647 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b64861-8fae-4a3f-a18f-f19546e33da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:12.118932 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b64861-8fae-4a3f-a18f-f19546e33da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:12.119207 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b64861-8fae-4a3f-a18f-f19546e33da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:12.119902 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b64861-8fae-4a3f-a18f-f19546e33da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:12.121534 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b64861-8fae-4a3f-a18f-f19546e33da6 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:12.131962 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1418/2886] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:50:12.135961 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d895ff-58e6-4ecd-adcc-f4dc887fa533 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:12.145803 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d895ff-58e6-4ecd-adcc-f4dc887fa533 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:12.151079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a4744-df55-4e91-a8a6-147668f17c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:12.151683 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4744-df55-4e91-a8a6-147668f17c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:12.151966 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4744-df55-4e91-a8a6-147668f17c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:12.152242 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a4744-df55-4e91-a8a6-147668f17c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:12.152931 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a4744-df55-4e91-a8a6-147668f17c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:12.154560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a4744-df55-4e91-a8a6-147668f17c29 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:12.165249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1469/2887] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Jul 28 06:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 06:50:12.169147 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b64861-8fae-4a3f-a18f-f19546e33da6 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:12.179174 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b64861-8fae-4a3f-a18f-f19546e33da6 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:12.183729 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:12.184383 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:12.184669 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:12.184945 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] PATH_INFO: `/v3/projects/81acdc7886e0448bb4a413ee34bc612a/users/e767b815733f4d708c57ef39d6516701/roles/551979dba2834cf397f2c386c2c64b2d` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:12.200306 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81acdc7886e0448bb4a413ee34bc612a, user_id=e767b815733f4d708c57ef39d6516701, role_id=551979dba2834cf397f2c386c2c64b2d)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:12.202982 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:12.216413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1419/2888] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Jul 28 06:50:12 2022] PUT /identity/v3/projects/81acdc7886e0448bb4a413ee34bc612a/users/e767b815733f4d708c57ef39d6516701/roles/551979dba2834cf397f2c386c2c64b2d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:50:12.226981 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2758-899e-4209-b030-7bab43cdce4a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:12.227277 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2758-899e-4209-b030-7bab43cdce4a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:12.227560 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2758-899e-4209-b030-7bab43cdce4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:12.255213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-16af2758-899e-4209-b030-7bab43cdce4a None None] MFA Rules not processed for user `e767b815733f4d708c57ef39d6516701`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:12.256943 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-16af2758-899e-4209-b030-7bab43cdce4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:12.321080 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1470/2889] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:50:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:12.331789 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:12.334250 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:12.355610 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:12.393839 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22298786-f5cc-4a93-aab5-d4edfe756f28 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:12.397696 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b9675-f521-4f2b-9f85-04915a7526c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6ee12d4109f4d8ba0d502558b9a2ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:12.398311 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b9675-f521-4f2b-9f85-04915a7526c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:12.398592 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b9675-f521-4f2b-9f85-04915a7526c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:12.398867 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b9675-f521-4f2b-9f85-04915a7526c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:12.407040 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b9675-f521-4f2b-9f85-04915a7526c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:12.408465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b9675-f521-4f2b-9f85-04915a7526c2 service cinder] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:12.417175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1420/2890] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:50:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:50:13.778845 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013cbdf-026c-4fd1-a234-a98c6c6eee51 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:13.779386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013cbdf-026c-4fd1-a234-a98c6c6eee51 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:13.779386 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013cbdf-026c-4fd1-a234-a98c6c6eee51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:13.798290 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3013cbdf-026c-4fd1-a234-a98c6c6eee51 None None] MFA Rules not processed for user `e767b815733f4d708c57ef39d6516701`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:13.799492 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3013cbdf-026c-4fd1-a234-a98c6c6eee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:13.806336 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1471/2891] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:50:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:22.413254 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=17 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:23.793611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:30.010407 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d7b3-3891-404e-916b-3030614d7de0 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:30.010931 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d7b3-3891-404e-916b-3030614d7de0 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:30.011055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d7b3-3891-404e-916b-3030614d7de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:30.072911 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-8210d7b3-3891-404e-916b-3030614d7de0 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:30.074149 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-8210d7b3-3891-404e-916b-3030614d7de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:30.139109 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1421/2892] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:50:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:30.186667 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013cbdf-026c-4fd1-a234-a98c6c6eee51 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:30.188046 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3013cbdf-026c-4fd1-a234-a98c6c6eee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:30.279478 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013cbdf-026c-4fd1-a234-a98c6c6eee51 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:30.284480 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a87f7-e210-4435-86fb-3c125a38aff9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:30.285099 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a87f7-e210-4435-86fb-3c125a38aff9 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:30.285410 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a87f7-e210-4435-86fb-3c125a38aff9 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:30.285686 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a87f7-e210-4435-86fb-3c125a38aff9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:30.294209 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a87f7-e210-4435-86fb-3c125a38aff9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:30.295775 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a87f7-e210-4435-86fb-3c125a38aff9 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:30.304088 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1472/2893] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:50:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:50:30.801159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb26b-73dd-4f5e-b86b-2d5ec2e8a678 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:30.801481 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb26b-73dd-4f5e-b86b-2d5ec2e8a678 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:30.801777 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb26b-73dd-4f5e-b86b-2d5ec2e8a678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:30.822247 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2f1eb26b-73dd-4f5e-b86b-2d5ec2e8a678 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:30.823438 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2f1eb26b-73dd-4f5e-b86b-2d5ec2e8a678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:30.830501 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1422/2894] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jul 28 06:50:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:30.842678 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a87f7-e210-4435-86fb-3c125a38aff9 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:30.853826 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a87f7-e210-4435-86fb-3c125a38aff9 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:30.858771 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20c870-6a1f-4b1d-ab2a-5798222f0780 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:30.859375 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20c870-6a1f-4b1d-ab2a-5798222f0780 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:30.859656 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20c870-6a1f-4b1d-ab2a-5798222f0780 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:30.859930 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20c870-6a1f-4b1d-ab2a-5798222f0780 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:30.869213 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20c870-6a1f-4b1d-ab2a-5798222f0780 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:30.870819 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20c870-6a1f-4b1d-ab2a-5798222f0780 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:30.878945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1473/2895] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:50:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:50:31.092954 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740633-59ac-4f98-b1b4-0567f560b05e None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:31.093465 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740633-59ac-4f98-b1b4-0567f560b05e None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:31.093586 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740633-59ac-4f98-b1b4-0567f560b05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:31.113319 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-07740633-59ac-4f98-b1b4-0567f560b05e None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:31.114505 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-07740633-59ac-4f98-b1b4-0567f560b05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:31.122229 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1423/2896] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Jul 28 06:50:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 06:50:31.134044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20c870-6a1f-4b1d-ab2a-5798222f0780 service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:31.144702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20c870-6a1f-4b1d-ab2a-5798222f0780 service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:31.150702 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4cbfe-a35f-42f1-8d56-222c5cf8b8fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:31.151342 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4cbfe-a35f-42f1-8d56-222c5cf8b8fa service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:31.151666 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4cbfe-a35f-42f1-8d56-222c5cf8b8fa service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:31.151988 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4cbfe-a35f-42f1-8d56-222c5cf8b8fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:31.160544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4cbfe-a35f-42f1-8d56-222c5cf8b8fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:31.162772 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4cbfe-a35f-42f1-8d56-222c5cf8b8fa service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:31.170846 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1474/2897] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:50:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:50:31.487180 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ffe34-a16a-406b-90bd-99b4a84f8ff4 None None] REQUEST_METHOD: `POST` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:31.487474 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ffe34-a16a-406b-90bd-99b4a84f8ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:31.487755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ffe34-a16a-406b-90bd-99b4a84f8ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:31.507410 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.auth.core [None req-2b4ffe34-a16a-406b-90bd-99b4a84f8ff4 None None] MFA Rules not processed for user `528c54678648461397f490fbbb698601`. Rule list: `[]` (Enabled: `True`). {{(pid=80276) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:31.508601 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-2b4ffe34-a16a-406b-90bd-99b4a84f8ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:31.515646 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1424/2898] 10.222.0.20 () {58 vars in 967 bytes} [Thu Jul 28 06:50:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:31.528221 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4cbfe-a35f-42f1-8d56-222c5cf8b8fa service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:31.538749 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4cbfe-a35f-42f1-8d56-222c5cf8b8fa service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:31.543652 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bee790-d4a0-4e99-804f-f9b6431fcb92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25dd5ceedc3040fdb75d9d24a78956a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:31.544249 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bee790-d4a0-4e99-804f-f9b6431fcb92 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:31.544527 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bee790-d4a0-4e99-804f-f9b6431fcb92 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:31.544800 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bee790-d4a0-4e99-804f-f9b6431fcb92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:31.552782 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bee790-d4a0-4e99-804f-f9b6431fcb92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:31.554473 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bee790-d4a0-4e99-804f-f9b6431fcb92 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:31.562538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1475/2899] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Jul 28 06:50:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 06:50:33.831809 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ffe34-a16a-406b-90bd-99b4a84f8ff4 None None] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:33.897123 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ffe34-a16a-406b-90bd-99b4a84f8ff4 None None] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:33.902071 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:33.902714 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:33.902995 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:33.903271 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] PATH_INFO: `/v3/users/e767b815733f4d708c57ef39d6516701` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:33.904454 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e767b815733f4d708c57ef39d6516701)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:33.906482 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:33.954085 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:33.955623 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e767b815733f4d708c57ef39d6516701'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:33.963658 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e767b815733f4d708c57ef39d6516701'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:33.964312 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e767b815733f4d708c57ef39d6516701'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:33.977159 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e767b815733f4d708c57ef39d6516701'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:33.977850 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e767b815733f4d708c57ef39d6516701'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:33.989755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1425/2900] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:50:33 2022] DELETE /identity/v3/users/e767b815733f4d708c57ef39d6516701 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:50:33.994868 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532efebf-4543-4cd4-bb20-8b980dde83e5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:33.995394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532efebf-4543-4cd4-bb20-8b980dde83e5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:33.995624 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532efebf-4543-4cd4-bb20-8b980dde83e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:34.014144 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-532efebf-4543-4cd4-bb20-8b980dde83e5 None None] MFA Rules not processed for user `5666224e1bf547bc87b970165a115158`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 06:50:34.015337 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-532efebf-4543-4cd4-bb20-8b980dde83e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:34.076889 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1476/2901] 10.222.0.20 () {58 vars in 958 bytes} [Thu Jul 28 06:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 06:50:34.087376 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:34.088787 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:34.152020 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572ee50-0ab8-4e06-93ca-680cad136f76 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:34.155678 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:34.156293 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:34.156579 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:34.156855 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:34.166852 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:34.168285 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:34.178074 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1426/2902] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:50:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:50:34.355697 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532efebf-4543-4cd4-bb20-8b980dde83e5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:34.357394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-532efebf-4543-4cd4-bb20-8b980dde83e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:34.369863 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532efebf-4543-4cd4-bb20-8b980dde83e5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:34.375611 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:34.376259 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:34.376544 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:34.376817 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] PATH_INFO: `/v3/projects/81acdc7886e0448bb4a413ee34bc612a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:34.378079 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] RBAC: Authorizing `identity:delete_project(project_id=81acdc7886e0448bb4a413ee34bc612a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:34.380824 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:34.413677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:34.431044 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81acdc7886e0448bb4a413ee34bc612a', 'request_id': 'req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:34.436645 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81acdc7886e0448bb4a413ee34bc612a', 'request_id': 'req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:34.444510 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81acdc7886e0448bb4a413ee34bc612a', 'request_id': 'req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:34.455859 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] The token cache is being invalidate because project 81acdc7886e0448bb4a413ee34bc612a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:50:34.456154 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:34.456799 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:50:34.458367 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1477/2903] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:50:34 2022] DELETE /identity/v3/projects/81acdc7886e0448bb4a413ee34bc612a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 06:50:44.173653 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=39,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:44.452844 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:50:53.071798 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:53.073236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:53.141689 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaebbe-f109-4957-87a7-5a9a112c6a3d service neutron] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:53.203042 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:53.203668 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:53.203959 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:53.204236 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:53.206026 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:53.218321 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:53.219916 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:53.231039 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1427/2904] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:50:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:50:54.990286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:50:54.991953 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:55.048897 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0b4c3-bde8-4a6b-bcc1-e346f8dc005f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:50:55.052330 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b46a15d481f4393bef8c4fa2281710b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:50:55.053134 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:50:55.053882 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:50:55.054155 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:50:55.055878 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:50:55.122268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:50:55.123696 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:50:55.134268 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1478/2905] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:50:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 06:51:01.463929 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:51:01.465557 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:51:01.480690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f4e5c-fb94-4380-8b14-912358759c1b service placement] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:51:01.485628 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:51:01.486373 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:51:01.486755 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:51:01.487110 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] PATH_INFO: `/v3/users/52586f7b50964ec7ae104a04e9be3004` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:51:01.488424 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] RBAC: Authorizing `identity:delete_user(user_id=52586f7b50964ec7ae104a04e9be3004)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:51:01.490491 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:51:01.536113 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52586f7b50964ec7ae104a04e9be3004'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.543455 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52586f7b50964ec7ae104a04e9be3004'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.544078 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52586f7b50964ec7ae104a04e9be3004'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.556009 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52586f7b50964ec7ae104a04e9be3004'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.556587 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52586f7b50964ec7ae104a04e9be3004'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.569361 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1428/2906] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:51:01 2022] DELETE /identity/v3/users/52586f7b50964ec7ae104a04e9be3004 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:51:01.745642 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:51:01.755394 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729cb94-6d23-43bd-aa6c-461e431faa42 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:51:01.812032 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:51:01.812597 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:51:01.812899 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:51:01.813177 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] PATH_INFO: `/v3/projects/e705818597064414b9d882e0dd2071c2` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:51:01.814358 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e705818597064414b9d882e0dd2071c2)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:51:01.816538 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:51:01.873945 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e705818597064414b9d882e0dd2071c2', 'request_id': 'req-962800a2-454c-4ee3-859d-4e274a4e0f19'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.875847 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:51:01.880979 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e705818597064414b9d882e0dd2071c2', 'request_id': 'req-962800a2-454c-4ee3-859d-4e274a4e0f19'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.889744 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e705818597064414b9d882e0dd2071c2', 'request_id': 'req-962800a2-454c-4ee3-859d-4e274a4e0f19'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.902286 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] The token cache is being invalidate because project e705818597064414b9d882e0dd2071c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:51:01.902602 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.903251 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:01.904949 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1479/2907] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:51:01 2022] DELETE /identity/v3/projects/e705818597064414b9d882e0dd2071c2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:51:01.909467 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:51:01.911055 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:51:01.921260 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:51:01.925413 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4753f9-d07c-4fe0-8715-fe51b17b754d admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:51:01.985832 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:51:01.986378 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:51:01.986690 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:51:01.986994 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] PATH_INFO: `/v3/users/f16df18818914189a4890f25355e43e0` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:51:01.988189 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16df18818914189a4890f25355e43e0)` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:51:01.990139 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:51:02.036730 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16df18818914189a4890f25355e43e0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.044019 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16df18818914189a4890f25355e43e0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.044638 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16df18818914189a4890f25355e43e0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.057202 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16df18818914189a4890f25355e43e0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.057878 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.notifications [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16df18818914189a4890f25355e43e0'} {{(pid=80276) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.071502 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1429/2908] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Jul 28 06:51:01 2022] DELETE /identity/v3/users/f16df18818914189a4890f25355e43e0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:51:02.242672 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:51:02.244237 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:51:02.258635 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962800a2-454c-4ee3-859d-4e274a4e0f19 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:51:02.321215 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5666224e1bf547bc87b970165a115158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25016d71f554431cbc0d0222ca313083', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:51:02.321954 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:51:02.322207 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:51:02.322483 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] PATH_INFO: `/v3/projects/b3f5d1fadd61488cba568e09f880dcba` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:51:02.323638 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f5d1fadd61488cba568e09f880dcba)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:51:02.325603 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:51:02.371035 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f5d1fadd61488cba568e09f880dcba', 'request_id': 'req-5da89e88-0d56-4053-8c62-0bd716a52412'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.376586 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f5d1fadd61488cba568e09f880dcba', 'request_id': 'req-5da89e88-0d56-4053-8c62-0bd716a52412'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.384646 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f5d1fadd61488cba568e09f880dcba', 'request_id': 'req-5da89e88-0d56-4053-8c62-0bd716a52412'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.395564 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] The token cache is being invalidate because project b3f5d1fadd61488cba568e09f880dcba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 06:51:02.395849 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.397194 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5da89e88-0d56-4053-8c62-0bd716a52412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 06:51:02.398763 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1480/2909] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Jul 28 06:51:02 2022] DELETE /identity/v3/projects/b3f5d1fadd61488cba568e09f880dcba => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 06:51:12.066869 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG dbcounter [-] [80276] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:51:12.392677 dv-cinder-850605-1-iscsi devstack@keystone.service[80277]: DEBUG dbcounter [-] [80277] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 06:56:14.556791 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] Authenticating user token {{(pid=80276) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 06:56:14.558298 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:56:14.700853 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a79a2-d565-4ae9-b132-fd7e8c65d774 admin admin] Validating token access rules against request {{(pid=80276) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 06:56:14.765204 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8956784-f7ea-41bb-82f9-781863dac922 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b18ac45ec948738129f3d08e0d0eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '473e18bb7e8542bbb3c23b7d813c583e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80276) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 06:56:14.765864 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8956784-f7ea-41bb-82f9-781863dac922 service placement] REQUEST_METHOD: `GET` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 06:56:14.766299 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8956784-f7ea-41bb-82f9-781863dac922 service placement] SCRIPT_NAME: `/identity` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 06:56:14.766585 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8956784-f7ea-41bb-82f9-781863dac922 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80276) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 06:56:14.768353 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.fernet_utils [None req-a8956784-f7ea-41bb-82f9-781863dac922 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80276) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 06:56:14.780695 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8956784-f7ea-41bb-82f9-781863dac922 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 06:56:14.782297 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8956784-f7ea-41bb-82f9-781863dac922 service placement] RBAC: Authorization granted {{(pid=80276) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 06:56:14.793175 dv-cinder-850605-1-iscsi devstack@keystone.service[80276]: [pid: 80276|app: 0|req: 1430/2910] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Jul 28 06:56:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)